9 ms·

Clojure at a Bank – Moving from Java (2012)

106 points by xvirk 12y ago

tormeh 12y ago
I always thought critical applications like banking needed compile time type checking. Clojure is neat and all, but it's dynamic.
- BjoernKW 12y ago
  Most of the times, they don't. Static typing helps with catching certain types of errors long before the compiled code arrives in production but it's just one tiny factor that contributes to overall software quality. Lack of expressiveness or lack of developer happiness can be much more detrimental to software quality than lack of static typing.
  - xfalcox 12y ago
    I second this. I'm currently creating a new dynamic website, which will see around 10k users a day. We could have done it in a week with rails scaffolding capabilities and the vast gem environment, but we are doing everything manually in Scala, because type safety and performance. It's so frustrating that I'm looking forward to quit very soon.
    - mercurial 12y ago
      This is a weird complaint. The fact that Rails has scaffolding and whatever Scala framework your team uses doesn't has little to do with type safety (there are plenty of web frameworks in dynamic languages without scaffolding as well). There are several languages which require a comparable amount of typing to a dynamic program, while adding considerable type safety.
    - scalesolved 12y ago
      The Play! Framework http://www.playframework.com http://www.playframework.com has been pretty sweet for us so far and we use Java with it and that plays second fiddle to Scala users who really enjoy the full benefit of the framework.
    - mod 12y ago
      I'd enjoy getting paid to go slow and learn Scala. I really despise the "let's finish this in one week" that is typically going on for me. I'd rather learn best practices, even if they're in a language, and have some experience building long-lasting services. Feels like the difference in working at a mobile home shop vs a custom log home company.
  - jeletonskelly 12y ago
    Exactly. Not to mention that in any large scale system your functional and integration tests are the most useful for producing high quality software than simply having static types.
    - the_af 12y ago
      No-one is arguing that you should just have static types and nothing else. The argument is that static types and testing are better than just testing. After all, static types are a kind of test that the compiler verifies for you, leaving you free to focus on the rest of the tests. I'd say in any large scale system, dynamic typing is a risk (though there are ways of mitigating this risk, of course).
  - mercurial 12y ago
    > Lack of expressiveness or lack of developer happiness can be much more detrimental to software quality than lack of static typing. Except 5 years later, when the happy developers have gone on to express themselves elsewhere, and left behind them a mess. Now, you can make a mess in any language, but a mess in a dynamic language is considerably harder to refactor. I also dispute the claim that a strong type system is a "tiny factor" in code quality. Being able to express invariants with types makes code much robust, and self-documenting.
    - liviu- 12y ago
      >I also dispute the claim that a strong type system is a "tiny factor" in code quality. There actually are some studies that show that using a static typing system is only a tiny factor when it comes to code quality: http://wadler.blogspot.co.uk/2011/09/experiment-about-static-and-dynamic.html http://wadler.blogspot.co.uk/2011/09/experiment-about-static... However, I've also read studies that show the opposite, but the lack of rigurosity and the possible confounds that show up for both sides seems to render this as an open question.
      - calibraxis 12y ago
        With Clojure, you can begin to compare static vs. dynamic typing — keeping the language the same.
        Glide 12y ago
        A language having static or dynamic typing is only a part of the puzzle. Having tools that let you understand a codebase better through static typing is more important. With a statically typed language you have the compiler as a baseline, but it enables IDEs to do far more powerful things.
      - mercurial 12y ago
        > However, I've also read studies that show the opposite, but the lack of rigurosity and the possible confounds that show up for both sides seems to render this as an open question. This doesn't look particularly convincing, indeed (though studies about programming languages rarely are).
  - lmm 12y ago
    > Static typing helps with catching certain types of errors long before the compiled code arrives in production but it's just one tiny factor that contributes to overall software quality. If you use them right, expressing the constraints of your system in the type system, you can improve huge areas of quality. It doesn't give you much for free, but it does give you a tool that lets you check your own correctness properties more efficiently and maintainably than any alternative. > Lack of expressiveness or lack of developer happiness can be much more detrimental to software quality than lack of static typing. Agreed - but good static typing makes a language more expressive, not less.
- iak8god 12y ago
  The arguments I've heard supporting this are usually weaker than their proponents think because they simply don't understand the alternative. Even if this is true though, "banking" is much too broad a category to make such general statements. Consider: > Here are some specifics – our new code is going to be an order of magnitude less in volume than the old and this is being conservative. Java's static typing would help catch mistakes but it's not magic. And by moving this piece of their system to Clojure, they've managed to simplify it so much that there's much less room for mistakes to hide.
  - thu 12y ago
    The verbosity of Java isn't necessary in all statically-typed languages. For instance Haskell is no more verbose than, say, Python, and has an even more expressive static type system than Java.
    - iak8god 12y ago
      I didn't mean to imply that the verbosity/complexity of their original code base was caused by the type system. The article is light on details, so we can only speculate: maybe they'd be better off with Haskell but just failed to consider it, or maybe they considered all options and decided it would be easier to retrain their team and integrate with the existing code based if they used a JVM Lisp. What is clear is that they have a much smaller and more maintainable code base now than they had before, and so they're better off than they were.
      - the_af 12y ago
        I agree with you that a smaller, cleaner codebase with less boilerplate probably leads to better software, regardless of type system.
- calibraxis 12y ago
  Ok, then use Prismatic Schema & core.typed. Not to mention other tactics to help correctness, like simulation testing with Simulant. (Or what don't you like about them?)
- kyleburton 12y ago
  Clojure supports strong typing if you want it: https://github.com/Prismatic/schema https://github.com/Prismatic/schema https://github.com/clojure/core.typed https://github.com/clojure/core.typed These are both libraries that you can choose to use. Both Typed Clojure and Schema are more powerful than Java's type system. By powerful I mean you can declare types and constraints that aren't expressible in most type systems (eg: an object can not be NULL, or a Map or Array must have specific keys or type'd values). Schema is run-time (we only leverage it in development and testing), while Typed Clojure is more akin to compile time.
  - shellac 12y ago
    I'm ignorant here. From what you say it sounds like Typed Clojure is traditionally what we think of as 'types', whereas schema is more along the line of asserts. Is that fair?
    - taylorlapeyre 12y ago
      That's correct.
    - mtrimpe 12y ago
      Yup; but asserts that a) describe an entire 'object' (usually a map or list) b) can be composed together and c) be used to generate e.g. documentation or validated forms
  - deleted 12y ago
    [deleted]
  - mike_hearn 12y ago
    By powerful I mean you can declare types and constraints that aren't expressible in most type systems (eg: an object can not be NULL, or a Map or Array must have specific keys or type'd values) Hmm. Can it do anything better than that? Java has optional nullity annotations that tools like Findbugs and (more usefully) IntelliJ can use to highlight nullity bugs. Kotlin has nullity integrated into the language's type system in a much better way. Stating that a Map or Array must have specific types in it is the whole purpose of generics, which Java had since 1.5, no? Don't get me wrong. The Java type system is not that strong and has some frustrating holes in it. But typed arrays and nullity tracking doesn't seem like some advanced Clojure-only tech, to me.
- noelwelsh 12y ago
  Technology adoption is almost always driven by the people, not the technology. The OP is a big fan of Clojure, having used it elsewhere, so its no surprise they brought Clojure to their new job. To be clear, I'm not suggesting that Clojure is a bad language. I really enjoy using Lisp-like languages! I also enjoy modern statically typed languages (and on balance prefer them). My experience is that having good people and avoiding bad technology choices (e.g. choosing Ruby for high throughput and low latency environments) is the secret to success. Stories like these are invariably sold as technology successes (and, equally, technology is blamed when things go bad.) I believe the reality is more that better technology attracts better people. It's the better people and chance to avoid legacy that gives success.
- weavejester 12y ago
  Java doesn't have particularly good compile-time type checking. It's mostly there for performance rather than safety guarantees. While Clojure is a dynamic language, it actually does have optional type checking, courtesy of the core.typed library. This isn't perfect, but it is considerably more sophisticated than Java's inbuilt type system. Even without static type checking, I'd argue that Clojure is the safer language by default, since it mostly avoids mutability.
  - HillOBeans 12y ago
    This. The immutable nature of Clojure is what, IMHO, makes it such a better choice than Java for me. I frequently see discussions/debates around type safety on HN, but in my experience, it has been the mutable nature of Java systems that has created more bugs than anything else. Clojure being immutable by default has drastically reduced the pain caused by these bugs. Clojure's collections, FP style, and meta-programming (via macros) all make it a great choice for me, but I think being immutable by default is the biggest thing.
- deleted 12y ago
  [deleted]
- robmcm 12y ago
  I know at least 3 of the largest banks hevily use Python http://www.reddit.com/r/Python/comments/1ohpyt/how_and_why_bank_of_america_decided_to_rebuild http://www.reddit.com/r/Python/comments/1ohpyt/how_and_why_b... Not to say they don't use Java as well various other languages, but right tool for the job...
  - vram22 12y ago
    Interesting. Bank of America was supposed to have gotten heavily into Python a couple of years (or more) ago. I had blogged about it here at the time: http://jugad2.blogspot.in/2013/10/bank-of-america-to-rebuild-tech-stack.html http://jugad2.blogspot.in/2013/10/bank-of-america-to-rebuild... and Niall O'Connor from the bank (who spoke at PyCon IE 2013 - http://python.ie/pycon/2013/speakers/niall_oconnor/ http://python.ie/pycon/2013/speakers/niall_oconnor/ ) confirmed via a comment on my post, that they were not "beginning" to do it, but that it had "already happened". Edit: The reddit thread quoted by the parent comment, also mentions BoA and Niall.
    - dodders 12y ago
      Goldman's core risk management system (SecDB) is written in python, as is JPMorgan's Athena and BofAs Quartz. I believe the main people responsible for SecDB implemented Athena at JP and then Quartz at BofA.
      - swedishturnip 12y ago
        SecDB is not written in Python (unless things have changed recently). However both Athena and Quartz are heavily influenced by SecDB.
        hga 12y ago
        It's written in their own Slang language: http://stackoverflow.com/questions/3392636/slang-goldman-sachs-proprietary-programming-language http://stackoverflow.com/questions/3392636/slang-goldman-sac... And the whole system is in part credited with helping them survive the recent financial crisis ... although simply listening to their risk people was more important. But it's all intertwingled, I'm sure, because good risk people would want a system like SecDB so they know the company's positions in quasi-real time with a great deal of assurance vs. hours and days and through e.g. scraping spreadsheets, often manually, etc.
- akmiller 12y ago
  I'd feel better building complex financial system in Clojure rather than C# (the 2 languages I'm most familiar with at this point). With contractual based programming, pre/post assertions, schema validation, etc... you can add all the type checking you want. That along with the other well known benefits of functional programming I think would make it highly desirable to build large complex systems in.
- dragonwriter 12y ago
  > I always thought critical applications like banking needed compile time type checking. They need quality, trustworthy software. Compile time type checking is a tool that can be used to help get there (but how useful it is in getting there depends on how robust the type system is, and Java's is not particularly robust). Clear and concise code that is readily understood, avoids visual noise so that the programmer can focus on function, also is a tool that can help with that. Clojure focuses more on the latter than the former, but that doesn't necessarily make it worse, and may make it better.
- rdtsc 12y ago
  Note that one of the largest and reliable systems -- smart phone to internet gateways often run Erlang, another functional and language with dynamic (but strong) typing. So when you watch cat videos on your smart phone, there is about 50% some non-statically compiled functional code has done its job in setting up the necessary signaling. WhatsApp has also recently been the poster child for running a successful distributed platform, with a tiny number of engineers (about 10 or 20) by also leveraging a non-statically compiled and functional language. In general static type checking is nice, I like it. But clearly not a deal breaker. O often hear "I won't touch anything unless it has static typing" or "large applications should have static types". But maybe the question to ask, should applications be that large? Lately we have been all upvoting microservices. And if anything those encourage decoupling large monstrous application into smaller independent services. At that point there are just a soup of dynamically typed components.
  - Ironballs 12y ago
    From what I've seen, Clojure has a strong type system, but is dynamic. These terms aren't orthogonal. If I understand correctly (+ 'a' 1) is not allowed. Immutability only strengthens it. I ask this because I've been eager to get my hands dirty with Clojure. I have experience with other functional languages (Haskell, OCaml, F# and some Emacs Lisp), so the paradigm isn't new. I've looked at core.typed, but doesn't seem as neat as the syntax of Typed Racket. It's something, though. Is it checked at compile time? What irks me is that static typing and type inference can make code feel really robust, and for the lack of a better word, safe. In Clojure I see a lot of nice things but the lack of typing, though common for Lisps, always bothers me a bit. I don't like runtime errors that happen because the compiler wasn't able to tell me that this object doesn't have that method. JavaScript's undefined is not a function or its kin in Python are examples of this behaviour, which I don't like. Though, I see that Clojure's answer to this is rapid REPL development--which is great!--and unit testing, and what I've recently discovered, pre- and post-conditions. But it still feels somehow inferior. I am torn between learning Scala or Clojure. Knowing OCaml and F#, Scala doesn't look that interesting, messy and multi-paradigm. Clojure appeals to me because its a Lisp and has one paradigm, but on the other hand I'm scared by runtime exceptions. Should I just ignore my trepidations and proceed?
- logfromblammo 12y ago
  I always thought this was what SPARK/Ada was for, along with one of the real-time OSes. But it turns out that languages with the stick that far up their backsides are only used for actually critical applications, like air-traffic control, medical implants, crypto libraries, and spacecraft/satellites. If a banking program fails, they can just ask for a mulligan from the central bank/government. They're just playing with people's life savings, not any actual lives, right? If they're going to suck up all the best brains in the industry anyway, they ought to be able to do SPARK/Ada. If I were making banking/trading software, I'd start with an OS that combined the security paranoia of OpenBSD with the deterministic performance of LynxOS, build some tools that validate source code by automated proof rather than testing, and hire a bunch of people smarter than me to build software that is secure, reliable, and profitable, in that order. No bank in existence would be crazy enough to pay me to try.
  - hga 12y ago
    Above a certain level I don't think your thesis is currently valid. For a "bank bank", "playing with people's life savings", the lower level transnational stuff, yeah. Although I don't get the impression that the current stuff doing that is prone to disastrous failures, and I believe the system has some slack here and there to reverse errors, i.e. transfers of money vs. buying and selling financial instruments. The current infamous failure in this area, which was WRT to the latter, was at Knight Capital. But above a certain level, at which point we're not so much "playing with people's life savings" (or, rather, if they have a clue, only a small portion of them), the situation can be a lot more fluid. E.g. changing trading strategies, which can be driven by the capriciousness of governments. The current state of the art doesn't make it practical to embed the US tax code into source code amenable to automated proof, especially before it changes again, right??
- JackMorgan 12y ago
  I've been trying to come up with a common definition for language safety, and in scoring both C# and Clojure, Clojure came out a much more "safe by default" language. It's incomplete, but here is where I'm gathering the data: https://github.com/steveshogren/blog-source/blob/master/source/_posts/2015-02-17-programming-language-safety-algorithm.markdown https://github.com/steveshogren/blog-source/blob/master/sour...
postit 12y ago
Brazilian Startup Nubank (https://www.crunchbase.com/organization/nubank https://www.crunchbase.com/organization/nubank) is also clojure based. They use datomic, spark ...
arh68 12y ago
See also: Clojure at a Newspaper! [1] This guy should write a Clojure: War Stories book. [1] http://www.pitheringabout.com/?p=1018 http://www.pitheringabout.com/?p=1018
d4n3 12y ago
I love this quote in the comments: > when you have REPL you are TDD’ing in a different way as opposed to constructing persistent unit-test classes that could potentially form a layer of cement. Lately I've been using the console for development in rails and javascript more than anything and it is a really good way to iterate on a solution quickly.
- mercurial 12y ago
  It's great for quick feedback. It's less good when your coworker breaks the build on a less-used codepath because there are no unit tests to run.
- hga 12y ago
  Bingo. Especially in a functional language, if you architect your code well, e.g. avoiding large (lengthy) functions. You write a function, test it in the REPL, and forget about it. Until, of course, someone hands it data you weren't expecting. But here Clojure helps you a bit with it's effort to provide consistent interfaces to its collection types, e.g. lists, vectors and maps. And the most expressive and fast to write code in, the more time you can devote to testing. I'll bet in practice this sort of TDD works very well, as long as your coders are diligent. For the projects I've done this way, including a couple in C using a very good but now defunct interpretive environment for it (CenterLine's CodeCenter/ObjectCenter), it's been superb. In an extreme case, I spent around a week programming in C the engine of a high speed document imaging scanning system, at which point I got the driver that let me speak directly to the scanner. The first time I tried it end-to-end it worked ... until I ran out of file descriptors (had forgotten to close() each file after writing out its contents). There was of course a whole lot of experience that made that feat possible, but I found it very impressive how the environment allowed me to write code with so few flaws (that was, in fact, the only major one I can remember once I declared a function finished and ready to try out for real).
zettke 12y ago
Note this was from 2012, and the author no longer works there - I wonder how this system developed in the intervening years?
- ExpiredLink 12y ago
  > I wonder how this system developed in the intervening years? It probably developed into a great opportunity for consultants.
moomin 12y ago
These days, Jon is one of the main guys at Juxt.Pro, a London-based Clojure consultancy. They're very good... :)
ABS 12y ago
Here's the video from his talk at EuroClojure 2012 https://vimeo.com/45130708 https://vimeo.com/45130708
lmm 12y ago
One of the reasons I went with Scala instead of Clojure is that you don't "just have to make a jump". You can write one class, in code that clearly corresponds 1:1 to what you'd write in Java, and the interop is smooth enough that other classes don't notice that this class is in Scala. Later on you (hopefully) are writing a very different style of code, more declarative and immutable and putting much more of the business logic into the type system. But it doesn't have to be a big-bang switchover; you can be productive from day 1, hour 1 even.
- sz4kerto 12y ago
  And what did you gain? What I found is writing Scala that is not striving for ~100% immutability and functional style does not bring much benefit. It is just a different syntax, that is nicer than Java on one hand, but also big and not straightforward on the other (in the C++ sense, so 'Scala' does not have a conventional coding style, it ranges from Java to Scalaz). Also, making core 'half-functional' does not bring any benefits in terms of composability, concurrency, reasonability (about the code). That's why I am pro Clojure.
  - lmm 12y ago
    > And what did you gain? All the functional goodness of easier to reason about code. A much more concise way to express common patterns, and a type system powerful enough to abstract away things that you have to repeat in Java. More dynamism than is possible to use safely in a dynamically-typed language, because I can write code that simply "does the right thing" in a principled, safe way that the reader can confirm by checking the types, whereas in something like Python it would be impossible for a reader to see whether that code was buggy. > It is just a different syntax, that is nicer than Java on one hand, but also big and not straightforward on the other Actually the syntax is often more straightforward than Java's. E.g. {} just means a block in a sense that's handled consistently everywhere in Scala, whereas in Java {} are required on a try/catch or a method but optional on an if/else. > making core 'half-functional' does not bring any benefits in terms of composability, concurrency, reasonability (about the code). Of course it does, or at least if it doesn't then the whole functional programming effort is a mistake. Any "pure" core is a) relative to some notion of equivalence that might or might not correspond to the things that are important to your particular problem and b) doesn't do anything without an impure interpreter. See e.g. http://conal.net/blog/posts/the-c-language-is-purely-functional http://conal.net/blog/posts/the-c-language-is-purely-functio...
jim_greco 12y ago
UBS has a bit of a reputation for this. Same thing happened with their US Equities tech in the mid 2000s: 1) Bring in new blood to improve a legacy platform 2) The new blood decides to throw out everything and do a complete rewrite 3) The new system is super successful because it can focus on a much smaller subset of problems 4) The bank runs into profitability issues and can no longer pay developers well 5) The good developers leave en masse before everything is completely rewritten. 6) New developers come in and spends the entire time fighting fires and patching the system instead of building new stuff.
- eternalban 12y ago
  6a) .. because instead of using a language designed in part to address turnover and difficulty of keeping avant garde coders in banal and soul sucking jobs, the management passed the business decision of "what tools do we use?" to the said workers.
  - hga 12y ago
    On the other hand, the language chosen might ensure high turnover of the sorts of coders you'd really like to keep, assuming you could ever attract them in the first place. Management has choices here, including simply making the best business will stay good enough, or even if it doesn't, prioritizing keeping an essential core of its technology people short of anything but completely closing down the unit. One thing I've noticed that's very common in the long term success or failure of high tech companies is whether they kept their core technologists. Compare Microsoft to Lotus and perhaps Ashton-Tate (dBase). Companies like banks may not appear to be "high tech companies", but their "production" is done with computer systems so....
    - eternalban 12y ago
      > On the other hand, the language chosen might ensure high turnover of the sorts of coders you'd really like to keep, assuming you could ever attract them in the first place. I'm sure Bell Labs never had that problem :) I was reading someone's PhD dissertation on Chill [1] and what was interesting to note was how many of the players ended up outsourcing their software development to 3rd parties. Software is hard. There other day I was talking with someone in tech management from a national chain (think malls). They are rolling their own commerce platform since "nothing" out there can satisfy their ecommerce requirements. I didn't say this upfront -- a potential client -- but the idea that this chain will actually get into serious software development, develop an A team, and keep them, is complete and utter wishful thinking. My advice to anyone running a non-tech company is own the data and the apis and off load the rest to a[n] actual tech company. [1]: http://web.bi.no/forskning/papers.nsf/0/42215f66b6282965c12578df003a14ff/$FILE/2011-08-Paulsen.pdf http://web.bi.no/forskning/papers.nsf/0/42215f66b6282965c125...
mattmcknight 12y ago
This article conflates OO and strong typing a little too much. These are too very different things, but agree that the combination can cause problems, not to mention that so many Java architectures are pathological in terms of struggling to find the code that actually does something.
spiralpolitik 12y ago
It would be great to see examples of the design/modeling of the system. I really like functional languages have done since my university days, but there seems to be a dearth of examples on how to actually lay such large projects out in Clojure/Scala etc. Having something like the J2EE Petstore (for all its sins) would go a long way to allowing more projects to make the switch than more articles on how great things are.

106 points by xvirk 12y ago

I always thought critical applications like banking needed compile time type checking. Clojure is neat and all, but it's dynamic.

BjoernKW 12y ago
Most of the times, they don't. Static typing helps with catching certain types of errors long before the compiled code arrives in production but it's just one tiny factor that contributes to overall software quality. Lack of expressiveness or lack of developer happiness can be much more detrimental to software quality than lack of static typing.
- xfalcox 12y ago
  I second this. I'm currently creating a new dynamic website, which will see around 10k users a day. We could have done it in a week with rails scaffolding capabilities and the vast gem environment, but we are doing everything manually in Scala, because type safety and performance. It's so frustrating that I'm looking forward to quit very soon.
  - mercurial 12y ago
    This is a weird complaint. The fact that Rails has scaffolding and whatever Scala framework your team uses doesn't has little to do with type safety (there are plenty of web frameworks in dynamic languages without scaffolding as well). There are several languages which require a comparable amount of typing to a dynamic program, while adding considerable type safety.
  - scalesolved 12y ago
    The Play! Framework http://www.playframework.com http://www.playframework.com has been pretty sweet for us so far and we use Java with it and that plays second fiddle to Scala users who really enjoy the full benefit of the framework.
  - mod 12y ago
    I'd enjoy getting paid to go slow and learn Scala. I really despise the "let's finish this in one week" that is typically going on for me. I'd rather learn best practices, even if they're in a language, and have some experience building long-lasting services. Feels like the difference in working at a mobile home shop vs a custom log home company.
- jeletonskelly 12y ago
  Exactly. Not to mention that in any large scale system your functional and integration tests are the most useful for producing high quality software than simply having static types.
  - the_af 12y ago
    No-one is arguing that you should just have static types and nothing else. The argument is that static types and testing are better than just testing. After all, static types are a kind of test that the compiler verifies for you, leaving you free to focus on the rest of the tests. I'd say in any large scale system, dynamic typing is a risk (though there are ways of mitigating this risk, of course).
- mercurial 12y ago
  > Lack of expressiveness or lack of developer happiness can be much more detrimental to software quality than lack of static typing. Except 5 years later, when the happy developers have gone on to express themselves elsewhere, and left behind them a mess. Now, you can make a mess in any language, but a mess in a dynamic language is considerably harder to refactor. I also dispute the claim that a strong type system is a "tiny factor" in code quality. Being able to express invariants with types makes code much robust, and self-documenting.
  - liviu- 12y ago
    >I also dispute the claim that a strong type system is a "tiny factor" in code quality. There actually are some studies that show that using a static typing system is only a tiny factor when it comes to code quality: http://wadler.blogspot.co.uk/2011/09/experiment-about-static-and-dynamic.html http://wadler.blogspot.co.uk/2011/09/experiment-about-static... However, I've also read studies that show the opposite, but the lack of rigurosity and the possible confounds that show up for both sides seems to render this as an open question.
    - calibraxis 12y ago
      With Clojure, you can begin to compare static vs. dynamic typing — keeping the language the same.
      - Glide 12y ago
        A language having static or dynamic typing is only a part of the puzzle. Having tools that let you understand a codebase better through static typing is more important. With a statically typed language you have the compiler as a baseline, but it enables IDEs to do far more powerful things.
    - mercurial 12y ago
      > However, I've also read studies that show the opposite, but the lack of rigurosity and the possible confounds that show up for both sides seems to render this as an open question. This doesn't look particularly convincing, indeed (though studies about programming languages rarely are).
- lmm 12y ago
  > Static typing helps with catching certain types of errors long before the compiled code arrives in production but it's just one tiny factor that contributes to overall software quality. If you use them right, expressing the constraints of your system in the type system, you can improve huge areas of quality. It doesn't give you much for free, but it does give you a tool that lets you check your own correctness properties more efficiently and maintainably than any alternative. > Lack of expressiveness or lack of developer happiness can be much more detrimental to software quality than lack of static typing. Agreed - but good static typing makes a language more expressive, not less.
iak8god 12y ago
The arguments I've heard supporting this are usually weaker than their proponents think because they simply don't understand the alternative. Even if this is true though, "banking" is much too broad a category to make such general statements. Consider: > Here are some specifics – our new code is going to be an order of magnitude less in volume than the old and this is being conservative. Java's static typing would help catch mistakes but it's not magic. And by moving this piece of their system to Clojure, they've managed to simplify it so much that there's much less room for mistakes to hide.
- thu 12y ago
  The verbosity of Java isn't necessary in all statically-typed languages. For instance Haskell is no more verbose than, say, Python, and has an even more expressive static type system than Java.
  - iak8god 12y ago
    I didn't mean to imply that the verbosity/complexity of their original code base was caused by the type system. The article is light on details, so we can only speculate: maybe they'd be better off with Haskell but just failed to consider it, or maybe they considered all options and decided it would be easier to retrain their team and integrate with the existing code based if they used a JVM Lisp. What is clear is that they have a much smaller and more maintainable code base now than they had before, and so they're better off than they were.
    - the_af 12y ago
      I agree with you that a smaller, cleaner codebase with less boilerplate probably leads to better software, regardless of type system.
calibraxis 12y ago
Ok, then use Prismatic Schema & core.typed. Not to mention other tactics to help correctness, like simulation testing with Simulant. (Or what don't you like about them?)
kyleburton 12y ago
Clojure supports strong typing if you want it: https://github.com/Prismatic/schema https://github.com/Prismatic/schema https://github.com/clojure/core.typed https://github.com/clojure/core.typed These are both libraries that you can choose to use. Both Typed Clojure and Schema are more powerful than Java's type system. By powerful I mean you can declare types and constraints that aren't expressible in most type systems (eg: an object can not be NULL, or a Map or Array must have specific keys or type'd values). Schema is run-time (we only leverage it in development and testing), while Typed Clojure is more akin to compile time.
- shellac 12y ago
  I'm ignorant here. From what you say it sounds like Typed Clojure is traditionally what we think of as 'types', whereas schema is more along the line of asserts. Is that fair?
  - taylorlapeyre 12y ago
    That's correct.
  - mtrimpe 12y ago
    Yup; but asserts that a) describe an entire 'object' (usually a map or list) b) can be composed together and c) be used to generate e.g. documentation or validated forms
- deleted 12y ago
  [deleted]
- mike_hearn 12y ago
  By powerful I mean you can declare types and constraints that aren't expressible in most type systems (eg: an object can not be NULL, or a Map or Array must have specific keys or type'd values) Hmm. Can it do anything better than that? Java has optional nullity annotations that tools like Findbugs and (more usefully) IntelliJ can use to highlight nullity bugs. Kotlin has nullity integrated into the language's type system in a much better way. Stating that a Map or Array must have specific types in it is the whole purpose of generics, which Java had since 1.5, no? Don't get me wrong. The Java type system is not that strong and has some frustrating holes in it. But typed arrays and nullity tracking doesn't seem like some advanced Clojure-only tech, to me.
noelwelsh 12y ago
Technology adoption is almost always driven by the people, not the technology. The OP is a big fan of Clojure, having used it elsewhere, so its no surprise they brought Clojure to their new job. To be clear, I'm not suggesting that Clojure is a bad language. I really enjoy using Lisp-like languages! I also enjoy modern statically typed languages (and on balance prefer them). My experience is that having good people and avoiding bad technology choices (e.g. choosing Ruby for high throughput and low latency environments) is the secret to success. Stories like these are invariably sold as technology successes (and, equally, technology is blamed when things go bad.) I believe the reality is more that better technology attracts better people. It's the better people and chance to avoid legacy that gives success.
weavejester 12y ago
Java doesn't have particularly good compile-time type checking. It's mostly there for performance rather than safety guarantees. While Clojure is a dynamic language, it actually does have optional type checking, courtesy of the core.typed library. This isn't perfect, but it is considerably more sophisticated than Java's inbuilt type system. Even without static type checking, I'd argue that Clojure is the safer language by default, since it mostly avoids mutability.
- HillOBeans 12y ago
  This. The immutable nature of Clojure is what, IMHO, makes it such a better choice than Java for me. I frequently see discussions/debates around type safety on HN, but in my experience, it has been the mutable nature of Java systems that has created more bugs than anything else. Clojure being immutable by default has drastically reduced the pain caused by these bugs. Clojure's collections, FP style, and meta-programming (via macros) all make it a great choice for me, but I think being immutable by default is the biggest thing.
deleted 12y ago
[deleted]
robmcm 12y ago
I know at least 3 of the largest banks hevily use Python http://www.reddit.com/r/Python/comments/1ohpyt/how_and_why_bank_of_america_decided_to_rebuild http://www.reddit.com/r/Python/comments/1ohpyt/how_and_why_b... Not to say they don't use Java as well various other languages, but right tool for the job...
- vram22 12y ago
  Interesting. Bank of America was supposed to have gotten heavily into Python a couple of years (or more) ago. I had blogged about it here at the time: http://jugad2.blogspot.in/2013/10/bank-of-america-to-rebuild-tech-stack.html http://jugad2.blogspot.in/2013/10/bank-of-america-to-rebuild... and Niall O'Connor from the bank (who spoke at PyCon IE 2013 - http://python.ie/pycon/2013/speakers/niall_oconnor/ http://python.ie/pycon/2013/speakers/niall_oconnor/ ) confirmed via a comment on my post, that they were not "beginning" to do it, but that it had "already happened". Edit: The reddit thread quoted by the parent comment, also mentions BoA and Niall.
  - dodders 12y ago
    Goldman's core risk management system (SecDB) is written in python, as is JPMorgan's Athena and BofAs Quartz. I believe the main people responsible for SecDB implemented Athena at JP and then Quartz at BofA.
    - swedishturnip 12y ago
      SecDB is not written in Python (unless things have changed recently). However both Athena and Quartz are heavily influenced by SecDB.
      - hga 12y ago
        It's written in their own Slang language: http://stackoverflow.com/questions/3392636/slang-goldman-sachs-proprietary-programming-language http://stackoverflow.com/questions/3392636/slang-goldman-sac... And the whole system is in part credited with helping them survive the recent financial crisis ... although simply listening to their risk people was more important. But it's all intertwingled, I'm sure, because good risk people would want a system like SecDB so they know the company's positions in quasi-real time with a great deal of assurance vs. hours and days and through e.g. scraping spreadsheets, often manually, etc.
akmiller 12y ago
I'd feel better building complex financial system in Clojure rather than C# (the 2 languages I'm most familiar with at this point). With contractual based programming, pre/post assertions, schema validation, etc... you can add all the type checking you want. That along with the other well known benefits of functional programming I think would make it highly desirable to build large complex systems in.
dragonwriter 12y ago
> I always thought critical applications like banking needed compile time type checking. They need quality, trustworthy software. Compile time type checking is a tool that can be used to help get there (but how useful it is in getting there depends on how robust the type system is, and Java's is not particularly robust). Clear and concise code that is readily understood, avoids visual noise so that the programmer can focus on function, also is a tool that can help with that. Clojure focuses more on the latter than the former, but that doesn't necessarily make it worse, and may make it better.
rdtsc 12y ago
Note that one of the largest and reliable systems -- smart phone to internet gateways often run Erlang, another functional and language with dynamic (but strong) typing. So when you watch cat videos on your smart phone, there is about 50% some non-statically compiled functional code has done its job in setting up the necessary signaling. WhatsApp has also recently been the poster child for running a successful distributed platform, with a tiny number of engineers (about 10 or 20) by also leveraging a non-statically compiled and functional language. In general static type checking is nice, I like it. But clearly not a deal breaker. O often hear "I won't touch anything unless it has static typing" or "large applications should have static types". But maybe the question to ask, should applications be that large? Lately we have been all upvoting microservices. And if anything those encourage decoupling large monstrous application into smaller independent services. At that point there are just a soup of dynamically typed components.
- Ironballs 12y ago
  From what I've seen, Clojure has a strong type system, but is dynamic. These terms aren't orthogonal. If I understand correctly (+ 'a' 1) is not allowed. Immutability only strengthens it. I ask this because I've been eager to get my hands dirty with Clojure. I have experience with other functional languages (Haskell, OCaml, F# and some Emacs Lisp), so the paradigm isn't new. I've looked at core.typed, but doesn't seem as neat as the syntax of Typed Racket. It's something, though. Is it checked at compile time? What irks me is that static typing and type inference can make code feel really robust, and for the lack of a better word, safe. In Clojure I see a lot of nice things but the lack of typing, though common for Lisps, always bothers me a bit. I don't like runtime errors that happen because the compiler wasn't able to tell me that this object doesn't have that method. JavaScript's undefined is not a function or its kin in Python are examples of this behaviour, which I don't like. Though, I see that Clojure's answer to this is rapid REPL development--which is great!--and unit testing, and what I've recently discovered, pre- and post-conditions. But it still feels somehow inferior. I am torn between learning Scala or Clojure. Knowing OCaml and F#, Scala doesn't look that interesting, messy and multi-paradigm. Clojure appeals to me because its a Lisp and has one paradigm, but on the other hand I'm scared by runtime exceptions. Should I just ignore my trepidations and proceed?
logfromblammo 12y ago
I always thought this was what SPARK/Ada was for, along with one of the real-time OSes. But it turns out that languages with the stick that far up their backsides are only used for actually critical applications, like air-traffic control, medical implants, crypto libraries, and spacecraft/satellites. If a banking program fails, they can just ask for a mulligan from the central bank/government. They're just playing with people's life savings, not any actual lives, right? If they're going to suck up all the best brains in the industry anyway, they ought to be able to do SPARK/Ada. If I were making banking/trading software, I'd start with an OS that combined the security paranoia of OpenBSD with the deterministic performance of LynxOS, build some tools that validate source code by automated proof rather than testing, and hire a bunch of people smarter than me to build software that is secure, reliable, and profitable, in that order. No bank in existence would be crazy enough to pay me to try.
- hga 12y ago
  Above a certain level I don't think your thesis is currently valid. For a "bank bank", "playing with people's life savings", the lower level transnational stuff, yeah. Although I don't get the impression that the current stuff doing that is prone to disastrous failures, and I believe the system has some slack here and there to reverse errors, i.e. transfers of money vs. buying and selling financial instruments. The current infamous failure in this area, which was WRT to the latter, was at Knight Capital. But above a certain level, at which point we're not so much "playing with people's life savings" (or, rather, if they have a clue, only a small portion of them), the situation can be a lot more fluid. E.g. changing trading strategies, which can be driven by the capriciousness of governments. The current state of the art doesn't make it practical to embed the US tax code into source code amenable to automated proof, especially before it changes again, right??
JackMorgan 12y ago
I've been trying to come up with a common definition for language safety, and in scoring both C# and Clojure, Clojure came out a much more "safe by default" language. It's incomplete, but here is where I'm gathering the data: https://github.com/steveshogren/blog-source/blob/master/source/_posts/2015-02-17-programming-language-safety-algorithm.markdown https://github.com/steveshogren/blog-source/blob/master/sour...

postit 12y ago

Brazilian Startup Nubank (https://www.crunchbase.com/organization/nubank https://www.crunchbase.com/organization/nubank) is also clojure based. They use datomic, spark ...

arh68 12y ago

See also: Clojure at a Newspaper! [1] This guy should write a Clojure: War Stories book. [1] http://www.pitheringabout.com/?p=1018 http://www.pitheringabout.com/?p=1018

d4n3 12y ago

I love this quote in the comments: > when you have REPL you are TDD’ing in a different way as opposed to constructing persistent unit-test classes that could potentially form a layer of cement. Lately I've been using the console for development in rails and javascript more than anything and it is a really good way to iterate on a solution quickly.

mercurial 12y ago
It's great for quick feedback. It's less good when your coworker breaks the build on a less-used codepath because there are no unit tests to run.
hga 12y ago
Bingo. Especially in a functional language, if you architect your code well, e.g. avoiding large (lengthy) functions. You write a function, test it in the REPL, and forget about it. Until, of course, someone hands it data you weren't expecting. But here Clojure helps you a bit with it's effort to provide consistent interfaces to its collection types, e.g. lists, vectors and maps. And the most expressive and fast to write code in, the more time you can devote to testing. I'll bet in practice this sort of TDD works very well, as long as your coders are diligent. For the projects I've done this way, including a couple in C using a very good but now defunct interpretive environment for it (CenterLine's CodeCenter/ObjectCenter), it's been superb. In an extreme case, I spent around a week programming in C the engine of a high speed document imaging scanning system, at which point I got the driver that let me speak directly to the scanner. The first time I tried it end-to-end it worked ... until I ran out of file descriptors (had forgotten to close() each file after writing out its contents). There was of course a whole lot of experience that made that feat possible, but I found it very impressive how the environment allowed me to write code with so few flaws (that was, in fact, the only major one I can remember once I declared a function finished and ready to try out for real).

zettke 12y ago

Note this was from 2012, and the author no longer works there - I wonder how this system developed in the intervening years?

ExpiredLink 12y ago
> I wonder how this system developed in the intervening years? It probably developed into a great opportunity for consultants.

moomin 12y ago

These days, Jon is one of the main guys at Juxt.Pro, a London-based Clojure consultancy. They're very good... :)

ABS 12y ago

Here's the video from his talk at EuroClojure 2012 https://vimeo.com/45130708 https://vimeo.com/45130708

lmm 12y ago

One of the reasons I went with Scala instead of Clojure is that you don't "just have to make a jump". You can write one class, in code that clearly corresponds 1:1 to what you'd write in Java, and the interop is smooth enough that other classes don't notice that this class is in Scala. Later on you (hopefully) are writing a very different style of code, more declarative and immutable and putting much more of the business logic into the type system. But it doesn't have to be a big-bang switchover; you can be productive from day 1, hour 1 even.

sz4kerto 12y ago
And what did you gain? What I found is writing Scala that is not striving for ~100% immutability and functional style does not bring much benefit. It is just a different syntax, that is nicer than Java on one hand, but also big and not straightforward on the other (in the C++ sense, so 'Scala' does not have a conventional coding style, it ranges from Java to Scalaz). Also, making core 'half-functional' does not bring any benefits in terms of composability, concurrency, reasonability (about the code). That's why I am pro Clojure.
- lmm 12y ago
  > And what did you gain? All the functional goodness of easier to reason about code. A much more concise way to express common patterns, and a type system powerful enough to abstract away things that you have to repeat in Java. More dynamism than is possible to use safely in a dynamically-typed language, because I can write code that simply "does the right thing" in a principled, safe way that the reader can confirm by checking the types, whereas in something like Python it would be impossible for a reader to see whether that code was buggy. > It is just a different syntax, that is nicer than Java on one hand, but also big and not straightforward on the other Actually the syntax is often more straightforward than Java's. E.g. {} just means a block in a sense that's handled consistently everywhere in Scala, whereas in Java {} are required on a try/catch or a method but optional on an if/else. > making core 'half-functional' does not bring any benefits in terms of composability, concurrency, reasonability (about the code). Of course it does, or at least if it doesn't then the whole functional programming effort is a mistake. Any "pure" core is a) relative to some notion of equivalence that might or might not correspond to the things that are important to your particular problem and b) doesn't do anything without an impure interpreter. See e.g. http://conal.net/blog/posts/the-c-language-is-purely-functional http://conal.net/blog/posts/the-c-language-is-purely-functio...

jim_greco 12y ago

UBS has a bit of a reputation for this. Same thing happened with their US Equities tech in the mid 2000s: 1) Bring in new blood to improve a legacy platform 2) The new blood decides to throw out everything and do a complete rewrite 3) The new system is super successful because it can focus on a much smaller subset of problems 4) The bank runs into profitability issues and can no longer pay developers well 5) The good developers leave en masse before everything is completely rewritten. 6) New developers come in and spends the entire time fighting fires and patching the system instead of building new stuff.

eternalban 12y ago
6a) .. because instead of using a language designed in part to address turnover and difficulty of keeping avant garde coders in banal and soul sucking jobs, the management passed the business decision of "what tools do we use?" to the said workers.
- hga 12y ago
  On the other hand, the language chosen might ensure high turnover of the sorts of coders you'd really like to keep, assuming you could ever attract them in the first place. Management has choices here, including simply making the best business will stay good enough, or even if it doesn't, prioritizing keeping an essential core of its technology people short of anything but completely closing down the unit. One thing I've noticed that's very common in the long term success or failure of high tech companies is whether they kept their core technologists. Compare Microsoft to Lotus and perhaps Ashton-Tate (dBase). Companies like banks may not appear to be "high tech companies", but their "production" is done with computer systems so....
  - eternalban 12y ago
    > On the other hand, the language chosen might ensure high turnover of the sorts of coders you'd really like to keep, assuming you could ever attract them in the first place. I'm sure Bell Labs never had that problem :) I was reading someone's PhD dissertation on Chill [1] and what was interesting to note was how many of the players ended up outsourcing their software development to 3rd parties. Software is hard. There other day I was talking with someone in tech management from a national chain (think malls). They are rolling their own commerce platform since "nothing" out there can satisfy their ecommerce requirements. I didn't say this upfront -- a potential client -- but the idea that this chain will actually get into serious software development, develop an A team, and keep them, is complete and utter wishful thinking. My advice to anyone running a non-tech company is own the data and the apis and off load the rest to a[n] actual tech company. [1]: http://web.bi.no/forskning/papers.nsf/0/42215f66b6282965c12578df003a14ff/$FILE/2011-08-Paulsen.pdf http://web.bi.no/forskning/papers.nsf/0/42215f66b6282965c125...

mattmcknight 12y ago

This article conflates OO and strong typing a little too much. These are too very different things, but agree that the combination can cause problems, not to mention that so many Java architectures are pathological in terms of struggling to find the code that actually does something.

spiralpolitik 12y ago

It would be great to see examples of the design/modeling of the system. I really like functional languages have done since my university days, but there seems to be a dearth of examples on how to actually lay such large projects out in Clojure/Scala etc. Having something like the J2EE Petstore (for all its sins) would go a long way to allowing more projects to make the switch than more articles on how great things are.