11 ms·

Clojure at a Real Estate Portal

156 points by dodders 11y ago

dmichulke 11y ago
I use an eerily similar stack with clojure, ring/compojure, http-kit, java.jdbc + c3p0, timbre and I have to say it's fantastic. I created a kind of a template with how to organize namespaces around that and how to address and automatically parse the URL/form parameters depending on the URL and send a "missing param" or "malformed param" back in the appropiate cases. It also deals with optional params The app basically only abstracts away the HTTP Server stuff and calls the appropriate call in an API namespace (where someone else could hook in if he wants to use his own server). Also, I now save all DDL in a separate resources/SQL folder and parse it if I need to rewrite the DB from clojure (e.g., at initialization after deploy). In the folder there is a schema.sql in which all other DDL is called via \ir. Checking whether the current version is the most up to date I still do manually (so if I add an index to some table in the DB, I add checking for this index via postgres native tables in the migrated? function) but this will be automated as well some time in the future. The reason why I use org.clojure/java.jdbc is because I can use all native postgres features (arrays, jsonb, tsvector, ...) without using the very weird java.sql constructors. This is also one of the major points against any current library.
- yenda 11y ago
  you should replace c3p0 by hikaricp https://github.com/brettwooldridge/HikariCP https://github.com/brettwooldridge/HikariCP
pka 11y ago
How did you handle refactorings? Like, changing userAddress's "type" from Address to Maybe Address? This is what scares me the most in Clojure and all other dynamic languages - that early on in the project, I'd make an unfortunate decision which I wouldn't be able to go back on once the project goes over 2-3kloc... without introducing hundreds of potential runtime errors, that is.
- mateuszf 11y ago
  One way is to use regression unit testing, though Clojure community is not very into testing.
  - jonpither 11y ago
    That last part is complete rubbish.
    - mateuszf 11y ago
      That's just what I've heard - sorry if it's incorrect.
      - calibraxis 11y ago
        Sadly, one must investigate original sources, otherwise we get soundbite-knowledge. Here's a blogpost written by a core Clojure contributor that starts: "Occasionally I hear someone say that the Clojure community is against testing." (http://tech.puredanger.com/2013/08/31/clojure-and-testing/ http://tech.puredanger.com/2013/08/31/clojure-and-testing/) Or take this discussion on testing: "Finally, testing is greatly facilitated by design. Ideal testing takes some design constraints, some specification and turns it into tests as opposed to sort of embodying design inside tests. That's inside out. But again, that's something we have to work more at. Stems like Quick Check are interesting because you're basically starting with propositions about your system, which reflect the design and saying you write the tests, computer." (https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/DesignCompositionPerformance.md https://github.com/matthiasn/talk-transcripts/blob/master/Hi...)
      - dkersten 11y ago
        The Clojure community is very big on testing, but not all of it is automated (eg testing on the REPL). Automated testing is popular too though, to the point where Clojure has a number of testing libraries (clojure.test, test.check, test.generative, expectations, speclj, midje).
pwm 11y ago
Semi off-topic, but might helps others as well: What should an experienced OO dev read to better understand how dynamic FP languages, like Clojure, alleviate the aforementioned cons of OO in the architecture of large-scale agile projects? In other words: I know from experience what's the downside of OO, but I don't know how dynamic FP languages would help without sacrificing the benefits of OO?
- Skinney 11y ago
  What are the benefits of OO? I normally think of encapsulation, but this isn't that big of a deal in Clojure (or many other functional languages) as you normally can't mutate objects anyway.
  - danenania 11y ago
    I see encapsulation as being more or less orthogonal to mutability. The main benefit is helping to preserve a consistent interface that doesn't expose much implementation structure. Functional languages are at least as capable of encapsulation as oo. It's just accomplished through namespaces instead of objects.
  - pwm 11y ago
    Hm, good question :). By the "benefits of OO" I meant a collection of strategies that evolved over time to tackle the difficulty of building complex systems: encapsulation, composition, interfaces, patterns, SOLID principles, etc... I think what I'm after is an OO-FP map of these concepts, or more probably: alternative strategies.
    - grayrest 11y ago
      I'll answer these specifically for Clojure since they vary by language. > encapsulation Encapsulation is primarily a state control concept. It reduces the surface area of a piece of data to the object it's contained in. The reduction in the amount of code that can touch the data makes it easier to reason about. Clojure works around this by having most of its data be immutable and most functions pure. Debugging is walking up the stack trace to find the change you didn't expect. > composition Pure functions compose trivially. Encapsulating data in a class actually limits your composition by design. Using maps lets you re-use the many functions in the standard library pretty much everywhere. > interfaces Clojure has an extended version called protocols. It provides an api guarantee but it can be retroactively implemented on Java classes you don't control. Look up the expression problem in clojure for considerably more depth. You can also use multimethods if you want to dispatch on something besides class. > Patterns I subscribe to the argument that patterns are workarounds for limitations in a language. Most languages have them but nobody else formalizes them like the Java people. There are ways to accomplish the same objectives, generally with less ceremony. > SOLID Single Responsibilty - Concept carries over to functions in a namespace, a good idea in general Open/Closed - I've always understood this to mean encapsulation, see above. Liskov substitution - Dynamic languages are duck typed. If you want more assurance you can add something like Schema, which does structual subtyping instead of nominal subtyping. Interface segregation - Preference in the community is for protocols to be VERY small, usually 1-5 functions. Dependency inversion - Tends to not come up. Usually when you're designing an API you have a good idea for when a consumer is going to want different behavior. In those cases, you have the API take a function instead.
khgvljhkb 11y ago
Thanks for the write-up! Juxt seems like a cool bunch - I'm not London-based (Berlin) but would not hesitate one minute given the chance to join them.
- jonpither 11y ago
  Hey, send us a mail to say hi at info@juxt.pro :-)
donjigweed 11y ago
The "Clojurians don't like testing" meme probably has more to do with Rich Hickey's famous "guard rail programming" [1] comment than anything else. Of course, even at the time, the joke within the community was, "Yes, Rich Hickey doesn't need to write tests....you do!" [1] http://www.infoq.com/presentations/Simple-Made-Easy http://www.infoq.com/presentations/Simple-Made-Easy (15:30)
- sheepmullet 11y ago
  > The "Clojurians don't like testing" meme probably has more to do with Rich Hickey's famous "guard rail programming" [1] comment than anything else. And Rich Hickey isn't against testing. He was having a jab at test driven design.
hackbinary 11y ago
If anyone is interested, he's talking about onthemarket.com. https://www.onthemarket.com/ https://www.onthemarket.com/ Not sure why he just didn't say it, that information is available elsewhere anyway. https://juxt.pro/ https://juxt.pro/ http://blog.juxt.pro/posts/otm.html http://blog.juxt.pro/posts/otm.html

156 points by dodders 11y ago

I use an eerily similar stack with clojure, ring/compojure, http-kit, java.jdbc + c3p0, timbre and I have to say it's fantastic. I created a kind of a template with how to organize namespaces around that and how to address and automatically parse the URL/form parameters depending on the URL and send a "missing param" or "malformed param" back in the appropiate cases. It also deals with optional params The app basically only abstracts away the HTTP Server stuff and calls the appropriate call in an API namespace (where someone else could hook in if he wants to use his own server). Also, I now save all DDL in a separate resources/SQL folder and parse it if I need to rewrite the DB from clojure (e.g., at initialization after deploy). In the folder there is a schema.sql in which all other DDL is called via \ir. Checking whether the current version is the most up to date I still do manually (so if I add an index to some table in the DB, I add checking for this index via postgres native tables in the migrated? function) but this will be automated as well some time in the future. The reason why I use org.clojure/java.jdbc is because I can use all native postgres features (arrays, jsonb, tsvector, ...) without using the very weird java.sql constructors. This is also one of the major points against any current library.

yenda 11y ago
you should replace c3p0 by hikaricp https://github.com/brettwooldridge/HikariCP https://github.com/brettwooldridge/HikariCP

pka 11y ago

How did you handle refactorings? Like, changing userAddress's "type" from Address to Maybe Address? This is what scares me the most in Clojure and all other dynamic languages - that early on in the project, I'd make an unfortunate decision which I wouldn't be able to go back on once the project goes over 2-3kloc... without introducing hundreds of potential runtime errors, that is.

mateuszf 11y ago
One way is to use regression unit testing, though Clojure community is not very into testing.
- jonpither 11y ago
  That last part is complete rubbish.
  - mateuszf 11y ago
    That's just what I've heard - sorry if it's incorrect.
    - calibraxis 11y ago
      Sadly, one must investigate original sources, otherwise we get soundbite-knowledge. Here's a blogpost written by a core Clojure contributor that starts: "Occasionally I hear someone say that the Clojure community is against testing." (http://tech.puredanger.com/2013/08/31/clojure-and-testing/ http://tech.puredanger.com/2013/08/31/clojure-and-testing/) Or take this discussion on testing: "Finally, testing is greatly facilitated by design. Ideal testing takes some design constraints, some specification and turns it into tests as opposed to sort of embodying design inside tests. That's inside out. But again, that's something we have to work more at. Stems like Quick Check are interesting because you're basically starting with propositions about your system, which reflect the design and saying you write the tests, computer." (https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/DesignCompositionPerformance.md https://github.com/matthiasn/talk-transcripts/blob/master/Hi...)
    - dkersten 11y ago
      The Clojure community is very big on testing, but not all of it is automated (eg testing on the REPL). Automated testing is popular too though, to the point where Clojure has a number of testing libraries (clojure.test, test.check, test.generative, expectations, speclj, midje).

pwm 11y ago

Semi off-topic, but might helps others as well: What should an experienced OO dev read to better understand how dynamic FP languages, like Clojure, alleviate the aforementioned cons of OO in the architecture of large-scale agile projects? In other words: I know from experience what's the downside of OO, but I don't know how dynamic FP languages would help without sacrificing the benefits of OO?

Skinney 11y ago
What are the benefits of OO? I normally think of encapsulation, but this isn't that big of a deal in Clojure (or many other functional languages) as you normally can't mutate objects anyway.
- danenania 11y ago
  I see encapsulation as being more or less orthogonal to mutability. The main benefit is helping to preserve a consistent interface that doesn't expose much implementation structure. Functional languages are at least as capable of encapsulation as oo. It's just accomplished through namespaces instead of objects.
- pwm 11y ago
  Hm, good question :). By the "benefits of OO" I meant a collection of strategies that evolved over time to tackle the difficulty of building complex systems: encapsulation, composition, interfaces, patterns, SOLID principles, etc... I think what I'm after is an OO-FP map of these concepts, or more probably: alternative strategies.
  - grayrest 11y ago
    I'll answer these specifically for Clojure since they vary by language. > encapsulation Encapsulation is primarily a state control concept. It reduces the surface area of a piece of data to the object it's contained in. The reduction in the amount of code that can touch the data makes it easier to reason about. Clojure works around this by having most of its data be immutable and most functions pure. Debugging is walking up the stack trace to find the change you didn't expect. > composition Pure functions compose trivially. Encapsulating data in a class actually limits your composition by design. Using maps lets you re-use the many functions in the standard library pretty much everywhere. > interfaces Clojure has an extended version called protocols. It provides an api guarantee but it can be retroactively implemented on Java classes you don't control. Look up the expression problem in clojure for considerably more depth. You can also use multimethods if you want to dispatch on something besides class. > Patterns I subscribe to the argument that patterns are workarounds for limitations in a language. Most languages have them but nobody else formalizes them like the Java people. There are ways to accomplish the same objectives, generally with less ceremony. > SOLID Single Responsibilty - Concept carries over to functions in a namespace, a good idea in general Open/Closed - I've always understood this to mean encapsulation, see above. Liskov substitution - Dynamic languages are duck typed. If you want more assurance you can add something like Schema, which does structual subtyping instead of nominal subtyping. Interface segregation - Preference in the community is for protocols to be VERY small, usually 1-5 functions. Dependency inversion - Tends to not come up. Usually when you're designing an API you have a good idea for when a consumer is going to want different behavior. In those cases, you have the API take a function instead.

khgvljhkb 11y ago

Thanks for the write-up! Juxt seems like a cool bunch - I'm not London-based (Berlin) but would not hesitate one minute given the chance to join them.

jonpither 11y ago
Hey, send us a mail to say hi at info@juxt.pro :-)

donjigweed 11y ago

The "Clojurians don't like testing" meme probably has more to do with Rich Hickey's famous "guard rail programming" [1] comment than anything else. Of course, even at the time, the joke within the community was, "Yes, Rich Hickey doesn't need to write tests....you do!" [1] http://www.infoq.com/presentations/Simple-Made-Easy http://www.infoq.com/presentations/Simple-Made-Easy (15:30)

sheepmullet 11y ago
> The "Clojurians don't like testing" meme probably has more to do with Rich Hickey's famous "guard rail programming" [1] comment than anything else. And Rich Hickey isn't against testing. He was having a jab at test driven design.

hackbinary 11y ago

If anyone is interested, he's talking about onthemarket.com. https://www.onthemarket.com/ https://www.onthemarket.com/ Not sure why he just didn't say it, that information is available elsewhere anyway. https://juxt.pro/ https://juxt.pro/ http://blog.juxt.pro/posts/otm.html http://blog.juxt.pro/posts/otm.html