14 ms·
Show HN: I wrote a book on implementing DDD, CQRS and Event Sourcing
- alex-lawrence 6y agoThe idea of this book is to provide a practical guide on how to implement DDD, CQRS and Event Sourcing. I chose to use JavaScript and Node.js as alternative to Java or C#, which are often used in comparable literature. After reading it, one should understand the fundamental concepts, how to apply them and how they can work in code. You can access a free sample of the book on the webpage in different formats. I worked on the book for four years as a side project. In 2016, I had worked in three different projects that applied either some or all of the concepts. The last one was the main motivating factor for the book. It was a startup that was settled on DDD, CQRS and Event Sourcing, even though the developers had limited knowledge. Together, we learned, experimented and applied as much as we could. We read "Domain-Driven Design" by Eric Evans, "Implementing Domain-Driven Design" by Vaughn Vernon and whatever we could find from Greg Young. With the knowledge we picked up and the experimentation we did, we felt in a good position. Still, there were many unanswered questions on how to put everything into code. In 2016, before leaving the startup, I first thought about writing a book. Effectively, I wanted to create the guide the startup would have needed in the beginning. When I started, I was overly confident about my knowledge. However, throughout the years, I learned many details. Today, I know that I could not have written it in 2016 at once. The final book is different from what I envisioned it to be at first. My initial goal was to write a short book with approximately 150 pages. The published version has now over 450 pages. However, I consider this a good thing. The book provides enough context to be read without any prior knowledge in DDD, CQRS or Event Sourcing. I've written a blog post that covers my motivation on writing the book in even more detail: https://www.alex-lawrence.com/posts/why-i-wrote-a-book-on-ddd-cqrs-and-event-sourcing/ https://www.alex-lawrence.com/posts/why-i-wrote-a-book-on-dd... I'd be happy to discuss your thoughts on and experiences with DDD, CQRS and Event Sourcing. Regardless whether it is about their combination, their sometimes counterproductive conflation or one of the concepts. Finally, if you have any input or feedback for me, it would be highly appreciated!
- AWebOfBrown 6y agoThis is interesting to me as someone who loves DDD but finds the tactical side hard to implement in Node. A few questions stand out: AFAICT implementing tactical DDD involves a lot of boilerplate code. That seems to be consistent with what you've written in your article on writing the book, mentioning "...we even built a Node.js framework as byproduct." If tackling DDD in Node requires so much work on something that isn't the businesses' core domain, and there's a lot of surface area for it to go wrong, why do it in Node? As someone who writes predominantly TypeScript, static types feel like a much lower hanging fruit for alleviating some of the pain in tackling rich domains. As you've written the book in JavaScript, I'm curious whether you used plain JavaScript on the professional projects? Also curious to hear from someone who has done tactical DDD in Java or C#: do you find a good supporting framework essential?
- alex-lawrence 6y agoCould you explain which aspects specifically require a lot of boilerplate code? Judging from my experience, I think the tactical DDD patterns are straight forward to implement. I didn't mention this in my post, but the Node.js framework was only concerned with CQRS & Event Sourcing. I fully agree with you that (static) types are inevitable for tackling rich/complex domains adequately. Personally, I would often recommend the use of TypeScript over plain JavaScript. As for the book, when I started working on it, I was using plain JavaScript. Over the course of the past four years, there were times, when I considered to rewrite the DDD parts with TypeScript. My current plan is to add an appendix or integrate additional content about (static) typing. While types are an important aspect for the Domain layer, I think the rest of the book works fine without TypeScript. Its absence might even help to keep the code example more concise. > I'm curious whether you used plain JavaScript on the professional projects? Yes, in both relevant projects we were using plain JavaScript (or CoffeeScript). We had many runtime type checks and a high test coverage to overcome the lack of static types. TypeScript would have definitely been the better choice.
- AWebOfBrown 6y ago> Could you explain which aspects specifically require a lot of boilerplate code? Probably the most frustrating part for me is tackling domain events and the implementation of the aggregate root. How have you tackled broadcasting domain events when an aggregate changes in a way that might be of interest to other bounded contexts?
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- JCDenton2052 6y agoDid you go with full event-sourcing? That is, not persisting domain objects at all, just reconstructing their state from the event store?
- alex-lawrence 6y agoIn one project, where we could have done otherwise, we went consciously full Event Sourcing. In another project, the Domain Model was inherently event-based. In my book, the Sample Application first persists domain objects and is later refactored towards Event Sourcing. While I read about the possibility of doing both at the same time, I never encountered this practice in a project. How I see it, there is always only one source of truth. If you apply Event Sourcing, it is the event log. Any other representation should be seen as derivation/projection. The one long-term project where we applied Event Sourcing showed quite some challenges you can face with this pattern. For me, the most challenging aspect is the versioning of an event-sourced system. When the Domain Model evolves, it might necessitate changes to events or the introduction of new ones. Greg Young even wrote a book on this topic: https://leanpub.com/esversioning https://leanpub.com/esversioning (I haven't read it though).
- JCDenton2052 6y agoCan you tell me a bit a more about the long-term project where you applied full ES? What kind of domain/business context informs this decision? I am aware of the costs of the approach, but would like to know the real world problems it solved for you.
- alex-lawrence 6y agoPlease note that I have limited experience with domains where Event Sourcing is without a doubt the weapon of choice. Hope my input still helps. Maybe there are other people that can give more qualified input on that. In the end, I am not trying to advocate for the pattern. The project was about building a SaaS for structured meetings, both for on-site and remote use. It involved topical areas such as collaboration, moderation and presentation. The model defined that every meeting consists of six steps, where each is facilitated with one specific method (out of multiple). There were also additional functionalities such as a collaborative whiteboard and audio/video communication. I briefly mentioned this in another comment: With regard to software architecture and pattern use, the project is a negative example. CQRS and Event Sourcing was prescribed on a system level. Regardless of conceptual/technological boundaries, every part made use of it. This decision was by no means reinforced with domain- or business-specific arguments. However, for us as developers, it was a great learning opportunity. In terms of significant benefits for specific contexts, I can only think of the whiteboard part. However, even for that, it is about a feature we never got to implement. At some point, we wanted to add a "Undo" functionality. With Event Sourcing, you can simply append events that represent the reversal of previously persisted state transitions. Independent of domain-specific aspects, I experienced three universal benefits. I hate to say it, but one was in fact reporting. Another benefit is the increased possibility to solve concurrency conflicts in a domain-specific way. Third, an event-sourced service is an ideal foundation for sending notifications, such as when publishing Domain Events. In the project, it helped us to build a real-time reactive User Interface. Of course, I'm not saying you necessarily need Event Sourcing for that.
- heybrendan 6y ago> Required technical knowledge > Working through the book requires advanced knowledge in JavaScript and in Node.js. Do you have any recommendations on various Javascript / Node.js education or training resources that would make good precursors to your book? Furthermore, any general advice for the uninitiated?
- colmvp 6y agoFor Javascript, I learned a ton from Frontend Masters. In particular for a deeper understanding of the language, Kyle Simpson's Deep Javascript Foundations where he connects the behavior of Javascript to the Standard. Overall, they have a number of high quality Javascript related courses.
- aethr 6y agoTry the "Modern Javascript Tutorial" https://javascript.info/ https://javascript.info/
- joshxyz 6y ago- mdn javascript for js basics - nodejs.org api docs for nodejs stuff - exploringjs.com for ES spec differences and progression - caniuse.com for checking browser support - there are bundlers like webpack and transpilers like babeljs but those are a little beyond the basics
- zeroc8 6y agoFor Javascript, I recommend "You don't know JS". I've tried tons of ressources and that's the one that finally made it click.
- serial_dev 6y agoI wouldn't recommend that book if the only reason you learn JavaScript is to be able to follow along this book (DDD, CQRS...). "You don't know JS" is a great book, and it's essential for someone who actually writes and works with JS all day. You'll pass most JS questions if you interview for a new job. At the same time, in my opinion, it's just a book full of gotchas that shouldn't even be present in good codebases (because most developers don't know JS :)).
- blackrock 6y agoHow big is the market for such a book?
- alex-lawrence 6y agoThat is a good question and, ideally, I should be able to answer it. Unfortunately, I don't know. The book started as a passion project and I never analysed the market in any useful way. However, I think that there is a certain potential for extending the existing market by making people interested in the topics.
- yeswecatan 6y agoJust browsed the table of contents and was both happy and surprised to see an entire chapter dedicated to the UI. How big is the sample application in the book? I've read a lot about DDD but things just don't seem to click for me, partly because example are trivial. For example, I read "Architecture Patterns with Python" (https://www.cosmicpython.com/book/preface.html https://www.cosmicpython.com/book/preface.html) but just didn't find it practical. Some random thoughts I had while reading, in no particular order: * what if domain logic can be optimized in the database? * how do joins across multiple tables work? * in Django I've come across plenty of examples where you have property method on a model that goes and performs additional queries. For instance, something along the lines of `my_post.get_latest_comment()`. Should this instead be somewhere in the service layer?
- Raz2 6y agoDon’t use django models in domain logic. I found it useful to map models to dataclasses inside DB layer and keep domain logic framework agnostic.
- alex-lawrence 6y agoThanks for the feedback. Happy to hear that you appreciate the UI chapter. For me, it was important to explain how to approach the User Interface. I often had the feeling that this part was a bit neglected when reading about CQRS & Event Sourcing. The book contains numerous standalone examples throughout the chapters, which are all executable. At the end of each chapter, the explained concepts are applied to the Sample Application context. The Sample Application is a greatly simplified task board. Users can register, login, create projects, manage team members and work with a task board. The task board is essentially a collection of items, grouped into three columns. Individual tasks contain details such as title, description, status, assignee, etc. While there are some constraints/invariants in the Domain Model, I would consider the Sample Application fairly trivial. It does not represent a complex domain problem. One reason for this is that I personally did not encounter any highly complex domain so far. It would have been weird if I tried to explain something I am not really experienced in. > what if domain logic can be optimized in the database? IMHO, the question is a bit too generic. The generic recommendation would be to not put domain logic into the database. However, one would need to look at what the domain logic in question is. Also, what does "in the database" mean? Are we talking about code that is being executed in the database environment, such as PL/SQL? > examples where you have property method on a model that goes and performs additional queries This sounds a bit like the Active Record pattern or the use of an ORM, where one "model" is somehow linked to another. Again, a generic answer would be that domain objects should stay free of infrastructural concerns. If you have posts and comments and need to get the latest comment on a post, this represents a use case. Use cases are executed by an Application Service. This part is allowed to query any number of domain objects / data sources. If you would be using CQRS, you would likely have a denormalized Read Model that contains comments per post.
- maxthegeek1 6y agoHave you looked into temporal.io/cadence workflow? They seem to implement the event sourcing pattern in a structured way.
- zeroc8 6y agoI'm in the midst of an event driven microservice project. On our first try, we've created a distributed monolith. That's why we decided to rewrite it in a more DDD driven way. I've started reading the blue book, the red book and Scott Millett's "Patterns and Principles of DDD". But the most helpful book I've found so far was "Domain-Driven Design in PHP", which I find a lot easier to grasp thanks to its hands on approach. That said, as a Go/Javascript person I really appreciate another book on the topic. I appreciate the effort and I am going to buy it.
- harrylucas 6y agoThis is really the big problem with DDD microservices. Whilst I'm a huge fan of the two, my go to is to always start out with a monolith and enforce boundraries within the application that could potentially become microservices at somepoint in the future. Even go as far as to use an in-memory event-driven system in the monolith. Then and only then when the need arises do we pull it out into it's own microservice. This has various benefits but the biggest one being you get to play with the design of the 'microservices' before actually committing to making them microservices allowing you to make better informed decisions. The harder part is that it does require stronger discipline and if not done correctly, a harder transition to microservices (e.g. pulling out the components)
- SergeAx 6y agoThis is great approach. Implementing well-designed horizontally-scalable monolith app usually leads to keeping it monolithic for years, saving enormous amount of time, money and effort, keeping devs team compact and agile.
- NicoJuicy 6y agoI'm actually doing exactly that, lol. Do you use c# ? I let only application layers talk to each other for now. Also, i have some issues finding a decent saga to use that stores it's state ( eg. Something like the library Mass transit)
- pc86 6y agoHow do you determine when "the need arises?" We've typically used bottlenecks, e.g. one piece of the app could benefit from additional scale while others wouldn't, and gone from there, but I feel like there's a better or more systematic approach that we're missing.
- 02020202 6y agogood for you. but as i have found out, reading about ddd/cqrs/es and actually implementing it is QUITE different. it sounds amazing but the amount of work that is required to implement it, in addition to the business logic, is staggering. as i have implemented few systems already, i am starting to think that ES is not really needed. i mean, it is needed in very specific use cases where knowing the exact state at the exact time is required(ie. finances), but 99.9999999% of apps will never need replay. so i am starting to lean towards change log instead of event log. the difference is that i can keep the simple crud logic without any overhead and simply log either simple entity diff as named event or go more granular and log changes like i'd do events but in simpler log way so a person(admin) can read what when where was changed. but no replay or browsing state changes on a single aggregate root. this is because i have come to a conclusion that if there is a problem in the future, human interaction is involved one way or another. so hard-coding everything will become a complication along the way(ie. fix by few click in UI or fix via code change + update script). on the other hand, with change log, admin can figure out the problem by looking at the changes that were made, make a call and perform the fix on the spot. no need to fiddle with code at all. just make few click, write some note and that is it. plus no code-based restrictions are preventing the admin to do it immediately - like input data validation or state transition validation and so on. also a big drawback of ES is keeping support for old events. in time, the code base will be growing and growing and you cannot throw the old code out.
- hurril 6y agoI am in the middle of setting the foundations of a new product project going DDD, CQRS and Event Sourcing so imagine my surprise when I see a book about... that. Then I see JavaScript and Node.js. So this isn't a book on either of DDD, CQRS or EventSourcing. It's about JS.
- alex-lawrence 6y agoIn fact, the book was called "Implementing DDD, CQRS and Event Sourcing in Node.js" up until the release of its complete version. I thought about this for a longer time and decided to remove the "in Node.js" part from the title. Instead, I made it clear in the book description. In my opinion, the book can be valuable for anyone who is interested in the concepts, regardless of technology choices. I think that it is fair to say that my book is partially also about Node.js, but not primarily. I wouldn't say though it is about JS. In fact, many of the code examples would look similar in other languages. Furthermore, the use of Node.js helps to provide concise code examples and executable code without much ceremony. When reading "Implementing Domain-Driven Design" by Vaughn Vernon, all the code examples are in Java. The book "Enterprise Integration Patterns" has a lot of code examples and all of them are also in Java. The most recent edition of "Refactoring" by Martin Fowler uses JavaScript. Still, I wouldn't say that these books are about a particular programming language.
- notretarded 6y ago>boo boo do my job for me >Then I see JavaScript and Node.js What's wrong with this? The underlying architectural patterns and abstractions you should be able to pick out and apply to your technology of choice.
- hurril 6y agoLook. My criticism would be the same with 8086 assembler as his language of choice. Yours too. Then where would we stand do you think? I'm just sick of people wasting a lot of energy on a sub-par language for things that could be of interest to a lot of people. People choose JavaScript to demonstrate monads for god's sake. Why? This is an: I have a hammer-strategy and that is what I am calling out. I write in a number of different languages (Java, Scala, F#, Haskell, C, C++, Rust, Idris, OCaml, Lisp, some dabbling in assembly) and they each have a profile when it comes to what they are able to present. The purer ML-dialects/ languages are a very good fit for compiler work. Java to show off, maybe, enterprisy and legacy patterns. Scala as a more familiar way of getting at pure functional programming for those that don't want to start off with Haskell. C++ for those of us that want to read about games. Rust for a lot of different things (that someone wants to re-write.) Idris to talk about some very advanced type systems in a way that is code rather than logic equations. JavaScript? Wtf does that present? How to use four layers of dependency resolution? How to make one page-applications with the highest number of enterprise patterns outside of J2EE anno 2001? It's not a serious choice.
- maxekman 6y agoAwesome! I’ll definitely get this. There are definitely numerous challenges with the approach, we’ll worth documenting and discussing! I’m the author of Event Horizon, a ES/CQRS toolkit for Golang: https://github.com/looplab/eventhorizon https://github.com/looplab/eventhorizon
- alex-lawrence 6y agoI will definitely look into this. What I can already say is that it has the best name for a ES/CQRS library!
- satyrnein 6y agoAnother architecture that seems to be becoming more common is having your CRUD functionality in a traditional monolith or a set of microservices, then writing events out or using change data capture to move that data into a data warehouse, where it is transformed for reporting purposes. While this is not the same as CQRS and event sourcing, it's a strategy that can be "bolted on" to your existing application and potentially capture some of the same benefits.
- yeswecatan 6y agoYup, that's how it has worked at my past couple of jobs. One question that always comes up is what should be in a UI vs. what should be in a reporting dashboard owned by the data team?
- stevenalowe 6y agoWell done, looking forward to reading this!
- alex-lawrence 6y agoThank you!