5 ms·
Ask HN: Best resources on abstracting and dealing with complexity
What would be your recommendations on resources that teach on how to abstract away and cope with complexity and hierarchy in technical domains?
- recursivedoubts 3y agoown-horn tooting, but... don't: https://grugbrain.dev https://grugbrain.dev
- Nidhug 3y agoI recommend David's Shapiro series on Systems Thinking that can be found here: https://www.youtube.com/@Systems.Thinking https://www.youtube.com/@Systems.Thinking It deals with more than just software, as it can be applied in many systems.
- sirwhinesalot 3y agoThe greatest advancement in computing, IMO, was structured programming. Ifs, Loops, Functions. What do they have in common? They got rid of goto spaghetti. Erlang actors excel in part because they are structured in supervision trees. There's that word again. If you can, structure things as a tree. If it is not possible, make a DAG. Only use arbitrary graphs as a last resort. This is why so many OO codebases turn into a mess, every object has references to who knows what, forming a complex dependency graph full of action at a distance.
- hiAndrewQuinn 3y agoUseful heuristic for me: Organize everything so that it doesn't require more than 4 chunks of short term memory to work with. https://hiandrewquinn.github.io/til-site/posts/the-rule-of-four/ https://hiandrewquinn.github.io/til-site/posts/the-rule-of-f...
- swah 3y agoOne thing that really works but I only did once or twice is throwing it out and starting again.
- o_nate 3y agoMark Seemann's book "Code That Fits in Your Head" has many practical ideas about how to manage code complexity with examples.
- max_ 3y agoTake Leslie Lamport's TLA+ course. [1] And learn TLA+. The core idea of TLA+ is that any problem, how ever complex can be represented as a state machine i.e a sequence of variables or states that change over time. It can be used to describe software, hardware and theoretical concepts like consensus algorithms. I suspect one low hanging fruit for TLA+ is to use it for describing physical phenomena gravity, magnetism, boiling etc. This I haven't seen work using it for that yet. [1]: https://lamport.azurewebsites.net/video/videos.html https://lamport.azurewebsites.net/video/videos.html
- golfinho23 3y agoThis is a manual solution. Create system sequence diagrams of your domain as you’ll learn a great deal in the process and have a visual heuristic to revisit when needed.
- ta988 3y agoThis is probably the most important thing I learned. Draw it down if it doesn't fit in your head. Keep your diagrams around, connect them, copy and paste them if they are made on a computer to make changes. Experiment.
- valand 3y agoNot software engineering exclusively, but this article in system domain in general https://donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/ https://donellameadows.org/archives/leverage-points-places-t...
- mnode 3y agoThis is great! Thanks for sharing.
- Layvier 3y agoIn software, "A Philosophy of Software Design" is a must read imo. Otherwise Rich Hickey's talks are always very insightful.
- __loam 3y agoI recommend this book to junior devs over "clean code". It's nice an short too, but dense with great advice.
- swah 3y agoMy favorites are here https://gist.github.com/rponte/c4a562f422974fd272dfe0598ebd83f9 https://gist.github.com/rponte/c4a562f422974fd272dfe0598ebd8...
- yuvadam 3y agoThis. By far the most important book any developer should read, no other book even comes close.
- vendiddy 3y agoThis has been my favorite book in over a decade. To me it felt like reading newton's laws for software development. Had a major impact on how I build software
- zabzonk 3y agoplease say why it is a must-read.
- __loam 3y agoExcellent advice on software development based on like 20 years of experience teaching software architecture at Stanford IIRC. It's really good, lots of specific advice, and not very dogmatic.
- EdwardCoffin 3y agoButler Lampson's 2020 expansion of his 1983 paper Hints and Principles for Computer System Design [1] [1] https://arxiv.org/abs/2011.02455 https://arxiv.org/abs/2011.02455
- whalesalad 3y agohttps://www.manning.com/books/grokking-functional-programming https://www.manning.com/books/grokking-functional-programmin... the concepts here are re: functional programming, but they apply to everything tbh. whether your system is a single monolith or a bunch of microservices, the core ideas hold true: think about your system in the form of actions (mutations), calculations/computations (reads), and data as data.
- taeric 3y agoI have lately been asserting that so much of computing interfaces is essentially materialized views into a database. Whether that database is relational or not really doesn't matter. Aggregation rollups and flat data entry are easy to see in that world with transformation from the flat data store that users directly write to into somewhat normalized backend systems that are all constantly running jobs to materialize the views that the rest of the systems use. To that end, I heavily stress that there is no single "data as data" view of any data in the system. It is all a transformation of some sort from somewhere. This is not to say that I disagree with your general statement. I would stress that the same "data" can have multiple representations in the "data" world. (This is not uncommon for outside of programming, either. Easy to consider that 1/3 is .33333... or 3.3...e-1, etc. Canonical formats for any data item are things you have to work to build, they do not come for free.)
- whalesalad 3y agoAt the end of the day, it is data. It might get read out of 3 different underlying systems, enriched by some kind of computation (dynamic or computed attributes vs ones that are actually persisted to a db, etc) but it is data. If you can think in data, as you say the underlying mechanisms for storing and retrieving that are less important. They are when it comes to performance and persistence guarantees, but if you abstract things correctly, that becomes easier. To add to my initial comment... Two things that come to mind as vital to a healthy system: SOLID principles (https://en.wikipedia.org/wiki/SOLID https://en.wikipedia.org/wiki/SOLID) and Clean Architecture (https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-a...), and Hexagonal Architecture (https://en.wikipedia.org/wiki/Hexagonal_architecture_(software) https://en.wikipedia.org/wiki/Hexagonal_architecture_(softwa...) come to mind. Recently I saw a post here that clean architecture was dead. This is hilarious to me and not true. If you are writing a low-level script, sure you can avoid that and optimize ... but for larger systems (ie, anything that is outside of a single process), clean+hexagonal is the way to go.
- 3np 3y agoYou don't mention where you're at. You're describing a major part of Software Engineering. Most reasonable university/college SE 101 courses will focus here. Which is to say, if you haven't (or it's been a while), check reading lists for university SE classes and pick the ones that focus on modeling. steer clear of process/agile material. Commonly used and recommended books would be GoF (if you're not familiar with the patterns already), Code Complete, and The Pragmatic Programmer.
- waynesonfire 3y agoSICP. I will never forgive my college professors for not teaching this to me. Many years later, when I discovered SICP on my own, I recognized that my high-school CS teacher wanted expose me to this. I was too young at the time to understand. Bummer. SICP is about managing complexity.
- namaria 3y agoOver the years I've come to realize that every single piece of advice or opinion in software is ultimately about complexity. Either about how to control and manage it properly or a good illustration of how easy it is to weaponize it and/or get overwhelmed by it.
- runningmike 3y agoI created my own guide after reading too much books: Simplify IT, The art and science towards simpler IT solutions , it’s cc-by on https://nocomplexity.com/documents/reports/SimplifyIT.pdf https://nocomplexity.com/documents/reports/SimplifyIT.pdf I also keep a very sort overview of imho good resources on Problem Solving Methods at https://www.bm-support.org/problem-solving-methods/ https://www.bm-support.org/problem-solving-methods/
- bionhoward 3y agoGreat question, I personally recommend: 1. Simple Made Easy by Rich Hickey (video https://www.infoq.com/presentations/Simple-Made-Easy/ https://www.infoq.com/presentations/Simple-Made-Easy/) 2. Thinking in Systems by Donella Meadows (pdf https://wtf.tw/ref/meadows.pdf https://wtf.tw/ref/meadows.pdf) 3. Raymond Hettinger’s content about CHUNKING is classic for this (one example video https://youtu.be/UANN2Eu6ZnM?si=TnLLj1CASwubkAS2 https://youtu.be/UANN2Eu6ZnM?si=TnLLj1CASwubkAS2) 4. The classic tome on Abstraction is Gödel, Escher, Bach: an Eternal Golden Braid, by Doug Hofstadter (https://en.m.wikipedia.org/wiki/Gödel,_Escher,_Bach https://en.m.wikipedia.org/wiki/Gödel,_Escher,_Bach) 5. Finally, I’ll cut myself at lucky number five by pointing you toward Jüergen (You-Again!) Schmidhuber’s fascinating webpage: https://people.idsia.ch/~juergen/ https://people.idsia.ch/~juergen/ Honorable mention to Margaret Hamilton’s T-Maps and F-Maps. Use a hierarchical numbered outline. Zoom in and out of the fractal network. Remember your working memory holds seven (7) concepts, plus or minus two (2) … so chunk size around five (5) or less is ideal for cognitive accessibility.
- paulorlando 3y agoTakes more of a humanities approach to technical domains, but there's https://unintendedconsequenc.es/ https://unintendedconsequenc.es/
- charlysl 3y agoPrinciples of Computer System Design This textbook, an introduction to the principles and abstractions used in the design of computer systems, is an outgrowth of notes written for 6.033 Computer System Engineering over a period of 40-plus years. Individual chapters are also used in other EECS subjects. There is also a web site for the current 6.033 class with a lecture schedule that includes daily assignments, lecture notes, and lecture slides. The 6.033 class Web site also contains a thirteen-year archive of class assignments, design projects, and quizzes. https://ocw.mit.edu/courses/res-6-004-principles-of-computer-system-design-an-introduction-spring-2009/ https://ocw.mit.edu/courses/res-6-004-principles-of-computer...
- riversflow 3y agoI swear I got this recommendation here, but I can't remember who to credit and can't find the post. Anywho, this is arguably the overarching reason Design Rules: The Power of Modularity[1] was written, at least in the abstract way you are asking it. The book tells you about the process that led to the creation of IBM's System/360, and it shows how a set of patterns and rules have converged in parallel industry sectors. It was honestly one of the most interesting books I've ever read, and that's exactly what it is about. Highly recommend this book to virtually anyone on this board. [1] ISBN10: 0262024667
- exikyut 3y agoPDF: copy-paste entire link into new tab or `base64 -d` encoded section data:text/plain;base64,aHR0cHM6Ly96bGlicmFyeS50by9maWxlZG93bmxvYWQvZGVzaWduLXJ1bGVzLXZvbC0xLXRoZS1wb3dlci1vZi1tb2R1bGFyaXR5LTA Not a converted EPUB, either. ^^
- simne 3y agoGood question. Good answers. I'll add one :) If considering not just software, but engineering or systems overall, best of all I seen is Goldratt, Theory of Constraints (usually named TOC). Resources (mostly books) are many, they even created their own accounting system and simulation software. For intro good books The Goal (1,2,3), they are for different markets and shows look on different aspect of TOC, and easy to read.
- zabzonk 3y agoPerhaps provide some links, and explain what "The Goal (1,2,3)" is? And who are "they"?
- simne 3y agoThe Goal Just 9" (20cm) https://www.amazon.com/Goal-Process-Ongoing-Improvement/dp/0566086654 https://www.amazon.com/Goal-Process-Ongoing-Improvement/dp/0... The Goal 2 https://www.amazon.sg/Goal-2-Eliyahu-M-Goldratt/dp/8185984107 https://www.amazon.sg/Goal-2-Eliyahu-M-Goldratt/dp/818598410... People name "The Goal 3" https://www.amazon.com/Necessary-But-Sufficient-Eliyahu-Goldratt/dp/0884271706 https://www.amazon.com/Necessary-But-Sufficient-Eliyahu-Gold... "The Theory of Constraints International Certification Organization (TOCICO) is an independent not-for-profit incorporated society that sets exams to ensure a consistent standard of competence" https://en.wikipedia.org/wiki/Theory_of_constraints#Certification_and_education https://en.wikipedia.org/wiki/Theory_of_constraints#Certific... Also exist lot of business organizations (official), who provide support on implementing TOC best practices, and sure, non-formal fan-clubs and lists of books and lists of other support files (learning software, accounting software).
- bob1029 3y agoOut of the tar pit changed how I look at everything. Relational modeling can be the ultimate answer for wrangling complexity. It supports real world messy things like circular dependencies without a hitch. You can achieve the same in OOP, but then go and try to serialize or persist an instance of such a thing... you will find a very unhappy variety of dragon lurking in that stack trace.
- davedx 3y agoOut of the Tar Pit.