27 ms·

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely sm

by daenz 4y ago

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coprocessor" assist in seeing the solutions. I believe there are well-known strategies (taken from the real world) for managing hierarchies of complexity that ease the cognitive load of exploring any problem. We just need to get these approaches into a programming "langugage"!

andrepd 4y ago
Can you elaborate and/or give examples? I read your comment and I can only see vague pseudo-profound statements.
astrange 4y ago
This is essentially Alan Kay/VPRI's old research program. I'm not sure why, but when people get into this they always think dataflow programming will solve everything - we already have Excel and it does do a lot, but it doesn't do everything.
papito 4y ago
It's a solved problem, if you are solving the right one. Most companies out there are solving problems they don't have. The FAANG disciples descended upon the world to preach the gospel of Complexity, and everyone bought it - because everyone wants to be like Google. What, you think "microservices" is a new invention? We called it distributed systems, and we really knew not to go there unless we wanted to decimate our productivity and sleep.
lbriner 4y ago
Good luck but your motivations will probably not line up with the masters who dictate things that become complexity. For example, I look at our software running on .net MVC and think as an Engineer, it simple and knowable but the Front-End Team are less worried about simplicity and more on flashy front-end stuff since that is their job. We end up bolting on a Front-end JS framework and complexity immediately ramps up by like 300%. Are they wrong for wanting a better and more flexible front-end? Not necessarily, I mean all the other companies have cool stuff and if we don't maybe our company dies. Ditto for lots of other examples...
chasd00 4y ago
this reminds me of the Head First series of books. I can still recall specific pages from the Head First Statistics book not because of the content but because every topic (and probably page) had a creative visualization.
rileyphone 4y ago
Me too, and I definitely agree that it is time to move past thinking in "languages" to a more holistic view of programming systems. The fact that as of now, only ~1% of people can access programming and wield the magic of the computer is a tragedy akin to the comparable levels of literacy in the medieval era. At the present, there seems to be a minor renaissance of thought along these lines stemming back to the pioneers of the field (McCarthy, Engelbart, Kay, etc). Everyone has a different take as to what the answer is, but I think "let 1000 flowers bloom" is the best way to find a solution to a complex and largely unknown problem such as software. Personally, I'm afraid that if we fail, the means of programming will become gated behind AIs and society will continue in its trance-like state of passivity instead of creative expression.
- notduncansmith 4y ago
  We will have to contend with the last bit regardless. Programming requires attention to detail and tolerance for tedium, the erosion of which will only continue as AI grows.
- wvenable 4y ago
  > The fact that as of now, only ~1% of people can access programming and wield the magic of the computer is a tragedy akin to the comparable levels of literacy in the medieval era. I don't know why programmers think this way. We don't expect everyone to be a doctor, a botanist, a novelist, or a musician but for some reason we think anyone can be a programmer. That's just not the case -- programming is a skill like any other -- it takes some natural inclination, some training, and a lot of practice. Just like any other skill. A programming language is to a programmer what an instrument is to a musician.
  - rileyphone 4y ago
    No, but we expect everybody to know reading, writing, and arithmetic. Programming may be more intrinsically difficult than these, but most of the reason it's so hard today is because of solvable problems that beginners run into, to the point of half of expertise is just knowing to avoid all these pitfalls. The downside of the open source developer ecosystem today is that putting it all together is typically a painful process, which is what Replit is aiming to solve. There is a world of difference between a good IDE and a bad one for a beginner, and a lot of that comes down to integration with the language. Anyways, there's a reason programming should be made more widely accessible - it develops thinking skills and rationality when you learn it, as Mindstorms pointed out. Between the anecdotes in the book and things like [0], I think it's a travesty that we aren't pursuing this to its fullest extent, and instead try to teach children Python or JavaScript, two decent languages for software development but not exactly forgiving with beginners. [0] https://medium.com/@stevekrouse/goodbye-seymour-cb712757264f https://medium.com/@stevekrouse/goodbye-seymour-cb712757264f
satyrnein 4y ago
I think I understand what you're getting at, but do you have any examples? Like you said, seeing helps understanding! :-)
- deleted 4y ago
  [deleted]
AnimalMuppet 4y ago
I don't think you can solve it. (Not a comment on you - I don't think anyone can solve it.) The problem is that software at least to some degree reflects the external environment. Let's say your software is in a financial company. Their software has to enable them to follow all the government financial regulations. Well, the government is following the larger societal "complexify to the point of collapse", and the financial regulations are certainly doing so. That means that the external behavior (the "business logic") of the software is insanely complex. You can't make that go away just by visual programming. But maybe you're not in the financial world. Maybe you're just writing programs for internal corporate processes at some generic company. Well, your software is still subject to the complexity that builds up in the company processes. Again, the programmers can't eliminate that complexity. Or maybe you're writing a customer-facing app - an external-facing web app, or an application that people actually install on their machines. Here you're at the mercy of the product or project manager trying to find new things for the app to do, and they still complexify the app to the point of collapse. The problem isn't that programming is too complicated. The problem is that what we want programs to do is too complicated. Visual programming can't save us from that.
perlgeek 4y ago
I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that convention. Storing both risks the data going out of sync. What's the "right" way to store person names? There doesn't seem to be a simple solution. Another example: we model physical cables (both for power grid and for data transmission) in our CMDB. All works fine, until you suddenly have a Y-shaped cable with three connectors that doesn't fit into your data model. The real world always has these 1% of cases that don't fit the general pattern; if you focus on the 99%, the 1% make trouble. If you focus on modeling every case, you have 10x the complexity, even for the simple case. And then there are things that are moderately complex and security critical, like password recovery workflows. We haven't really found a way to reuse these among different technologies. Like, if you once figured out the perfect password reset workflow with Ruby on Rails, and your next job uses Python + Django, you're back to square one. If somebody has a good idea for how to tackle these problems, please let me know!
- nightski 4y ago
  As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the same problems is insane. But this is not an easy problem to solve and I don't intend to trivialize it. I think we need to come up with better tools. code sharing through libraries/repositories (like npm) is great, but it can't be the final solution. Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. You could import modules, functions, data structures, anything. But that vision is still a long ways off.
  - feoren 4y ago
mamcx 4y ago
I think the same, but that requiere resources. I try to work on the side in a spiritual successor of Foxpro/DBase (https://tablam.org https://tablam.org). I consider a mix of relational/array model fill a lot of ergonomics for this (and it was proved to be right by the family of DBase langs). But what makes this much more complex today is the explosion on OS targets (Windows, Linux, MacOs, Android, iOS, Web), and the requirement to integrate with many other stuff, dealing with many formats (json, xml, ...), is harder to do UIs now and the base support is more inconsistent and mixed as ever... So, is possible to make a simpler tool, I certain of it, but then the developer/user will say "ah, ok, so how this connect to Redis, GraphQL and Amazon Web Services, run this on Android, Windows, parse CSV, ..." and that is what make this very hard at the end...
michaelbuckbee 4y ago
What you're describing is one approach, the other is that you pay a bunch of SaaS/PaaS/IaaS companies to handle a lot of the workload for you so you and a small team can just focus on the business logic of your particular niche.
- yhoiseth 4y ago
  Agree that this helps in some ways, but it also adds complexity. You can e.g. subscribe to a billing SaaS, but then you have to link your system to the billing system somehow
mathgladiator 4y ago
Welcome to the club! The fundamental problem of simplifying software is humans. Just consider date formats, time zones, and tax codes. Humans love to make complicated things. My philosophy on this has been to take the complex human stuff and stick it in a black box. A professional feather in cap with this approach is called BladeRunner ( https://dl.acm.org/doi/10.1145/3477132.3483572 https://dl.acm.org/doi/10.1145/3477132.3483572 ) which radically simplified the distributed system aspect by putting all the gnarly glue and business logic in a V8 VM (JavaScript). My next thing is a focus on board games where I have invented a programming language and have started to evolve a platform. It's called Adama ( https://www.adama-platform.com/ https://www.adama-platform.com/ ), and I think it is pretty cool. The interesting thing is that the complexity of board games is exceptional. I have a few clues to share. The first is that reactivity, which is found in excel, is a key to simplifying software as this makes the glue more automatic. Another clue is figuring out bidirectional communication which relates to reactivity as a two-way street. However, this is primarily hard because we don't have great things off the shelf to deal with this beyond TCP. For more of a deep dive, check out https://www.adama-platform.com/2021/12/22/woe.html https://www.adama-platform.com/2021/12/22/woe.html which talks about WebSocket. My final clue is that you can't run away from state. So many people offload state because state is hard, and you have to contend with it. I'm building yet another database.
- ozim 4y ago
  If you look from perspective of software creator that wants to handle all kind of timezones or post codes - yes humans complicate things. If you look from a person or user perspective - time zones or date formats are used in one location or in some context. People living in that context have it a lot easier because most of the time they don't care about other time zones. I would say people simplify things but on local scale. If you want to go global that is your problem not humans that live in one place and use single time zone all their life.
swatcoder 4y ago
This is an admirable and worthy pursuit, but keep in mind that any process you make streamlined: 1. becomes a post from which clever and ambitious people can build complex things. You invent docker to simplify application deployments and then somebody builds a n-dimensional microservice cloud on one side and starts commercializing new hardware architectures on the other. You don't remove the complexity, you just let it move around into new domains. (not a bad thing!) 2. is more temporal than you expect. The "finite and fundamental" problems of yesterday, today, and tomorrow are of different sets -- partly because (1) opens up new problem classes and partly because most problems are inescapably cultural and therefore subject to fashion cycles.
davidgay 4y ago
> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle. So in that view, you can't succeed at making software engineering always simple. Instead, you can make previously intractable problems tractable.
- daenz 4y ago
  If history is a guide, then you are correct. But I believe complexity is fractal (I think we all know this: "turtles all the way down..."), and that you can structure and manage complexity with recursive rules, so that you only see the resolution that you care about, for the areas that you care about, and everything else is a low frequency representation. Frameworks approximately do this, but for specific domains: they let you organize complexity into well-defined areas and put a pin in them, easing the cognitive load. Then you can handle those abstractions more easily. But yes, to your point, because frameworks typically only tackle a few levels of complexity, you still get that complexity back when you use the framework to advance the problems to the edge of what your framework is designed to address. A recursive/fractal management of complexity will allow all levels of the hierarchy to feel similar, so you are never increasing complexity, only looking at a different resolution. I think the key to this is mapping out the fundamental organizational problems and how they relate to each other at different resolutions.
  - ItsMonkk 4y ago
    > tackling more complex problems This is the 'induced demand' argument. As perfection seems impossible, and mistakes inevitable, this seems likely. However, the more I look into issues, the more I realize that a very small number of errors introduced early on is what ultimately causes a plethora of them. Just fixing a very small amount of these mistakes would have untold effects on computing over the long-term. That is, if we can get over the initial switching costs. > small team of engineers It took 2 men to over-take thousands creating Unix over Multix. It took Linus Torvalds - just about alone - a week to create git. We have already seen this prophecy come true. As teams get bigger, communication sales factorially, and the more people you have the more mistakes you make, which increases the size exponentially with each further mistake. What Unix and git showed is that when you put everything into a small team of engineers heads, they can work through the complexity enough until they can do it themselves. > Frameworks One of the things I realized a while ago is that pure/impure and library/framework have a decent mapping between the two. If you have a framework, you give it code and it acts for you, just like impure code. And so the problem we keep running into, is instead of inverting the flow like 'hexagonal architecture' says, we keep piling impure onto impure onto impure. Hexagonal says to not do anything of substance, keep the adapter clean, but each framework sure is doing something. Each layer on the stack we go up, the harder it is to get down. So now we have OS and applications and containers and k8s running micro-services that ends up being run in a web browser, when all we really needed was microkernals.
gtowey 4y ago
> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I think one thing missing from this line of thinking is that these are people. They have their own motivations and egos. They get sick, have families, take vacations, quit. I think part of the way large teams and large companies are structured is a hedge against changing teams with highly variable capacity. It's hard to overstate how much it can hurt a project to lose a someone who has a ton of experience locked up inside their head. The alternative is that I have worked for places that try to treat thier engineers like fungible tokens that can be shuffled and replaced at will. That environment feels extremely dehumanizing and demotivating.
randomsearch 4y ago
I agree with you. I have other things to do, but good luck to you. There’s an awful lot of cultural baggage in coding. Many of the concepts that seem essential - powerful text editors, devils tooling - can be completely removed. It requires rethinking from first principles and being willing to upset the Apple cart, but I believe it is possible.
gjvnq 4y ago
I think that's an honourable but unattainable persuit. A huge part of the problem is the messiness of the real world and costs. While there may be a finite set of fundamental problems, the set of possible hardware and software platforms is ever increasing and each has it's own constraints and strengths. A "universal programming environment" would need something like a "universal hardware interface". That's why things like Java and the Web became so popular despite their poor design. Also, visual programming is a hard thing to make work and human language skills seem to be far greater than human visual skills. Perhaps the best that can be done is "starting from zero" and making sure everything, from the ICs in your hardware to the memory models behind your software is thoroughly tested and formally verified. This is insanely expensive with current technology. Perhaps some fancy math and AI advancements can make formal verification powerful enough for ubiquitous use. Until then, I see little hope for a "universal programming environment".
kansface 4y ago
In addition to the comments below, complexity in software is just as often driven by politics/organizational structure. There is no solution in software to that problem.
- diamondap 4y ago
  Yes. And that's the root of this problem: > Nobody can subtract from the system; everyone just adds.
jandrewrogers 4y ago
This assumes that complex software problems are decomposable into discrete components defined and analyzable solely by their public interfaces. Complex software frequently cannot be modeled in this way -- it is what makes the software "complex". In reality, components have complex interactions far outside of what is captured in the component interfaces (see also: Hyrum's Law). Surfacing these implicit interactions at the component interface level just creates insanely complex interface definitions, moving the complexity somewhere else. To tie it to something concrete, just about everyone thinks C++ template metaprogramming is unreasonably difficult to reason about even aside from the syntax, but it exists to be extremely powerful at precisely expressing behavioral contracts of a software component that are difficult to do in other languages. Even then, it barely scratches the surface of what would be useful to express for the interaction of those components to be "simple". The number of design parameters of a component that really matter in various contexts are astronomical. No one can deal with reasoning about that many component traits in practice, so the software engineers simply hide most of them -- the complexity is still there and will manifest in unexpected ways because it is not visible at the interface. All systems engineering is complex for this reason. Any non-trivial software system has to be reasoned about as a monolith at some level to correctly address complexity, which creates an unavoidable cognitive load. This isn't a software engineering problem, it is a systems-thinking problem. In chemical engineering, for example, there are often complex system design problems that cannot be adequately addressed by decomposing them into a sequence of sub-problems, all that does is hide major complexity at the interface of the sub-problems.
- bigbonch 4y ago
  There are many embellished, hand wavy statements in this comment. Could you give real world examples of: * "In reality, components have complex interactions far outside of what is captured in the component interfaces" * How template metaprogramming "precisely express[es] behavioral contracts... that are difficult to do in other languages" I think you contradict your poetry with "Any non-trivial software system has to be reasoned about as a monolith at some level to correctly address complexity". If _any_ non-trivial software system _can_ be reasoned about as a monolith then it is true there exists some model with sufficient simplicity that there are well-defined discrete components.
milkanic 4y ago
Have you looked at Clojure and REPL/data driven development?
meheleventyone 4y ago
Isn’t this the dream since year dot? The problem is the real world is full of leaky abstractions, with different layers evolving at different rates and massive adoption inertia.
nradov 4y ago
I am skeptical of visual programming in general. Some of us just aren't visual thinkers. I find it much easier to conceptualize complex concepts in terms of text and narrative.
- tasuki 4y ago
  Don't worry - I am a visual thinker and I'm skeptical of visual programming. While it's cute for toy examples, I haven't seen it handle complexity well.
go_elmo 4y ago
I am sure every SE agrees that somehow things are always the same. How about making code pieces algebraic in order to reason and write with them much easier? Like using Haskell?
pebcakID10T 4y ago
I have been developing software for over 20 years and leading teams of developers for the last 10. Please allow me to give you some insight from experience. To your first point, in my experience, the best software, including complex platforms that handle massive amounts of traffic and data, can be built and maintained by small teams. I mean less than 20 developers. The larger the number of developers working on an application or platform has an almost inverse correlation to the speed in which new features are built or bugs fixed. Software architecture has been done visually since perhaps its inception (tools like UML). At most places I've worked, every new project or large feature is diagrammed visually to use as a guide in breaking down the project into component parts. In my experience complexity arises, not from lack of tools or industry knowledge, but from 3 main causes: 1. Inexperienced developer asked to create project, who just starts building without planning beforehand. 2. The main one - business demands features built that were never expected or planned for, and built as fast as possible. This causes developers to take shortcuts, make inelegant and difficult to maintain design choices. And leads to often inscrutable code that becomes technical debt especially after the original developer leaves the company. This will always happen as long as software is used to make a business money. 3. High team turnover - I've seen places where developers came and went so often that there was a myriad of things half started and never finished. How I've solved or helped alleviate these issues: Make the business case to company owners or management that technical debt will be an ever increasing impediment to development velocity and the dev team will need a percentage of work in any given sprint to tackle tech debt issues (as opposed to having everybody work 100% on new features and bug fixes all the time). I have successfully taken a platform that was bug-ridden, difficult to maintain, and where new feature development had slowed to a crawl due to the over-complexity of the software, to a place of stability and ease of development, simply by allowing our team to chip away at tech debt over the course of several years. Tech debt issues were rated by level of complexity, risk to the business in change (regression bugs), and impact on team velocity. We worked on the highest impact, lowest risk items first and kept going until there wasn't much left on that list.
throw827474737 4y ago
How many clever computer scientist (including me ) had this idea and vision at least once? And how many attempts by languages/DSLs/visual languages have already been tried to create? I believe you are wrong, I'd wish you are right ;)
dahart 4y ago
A very worthy, laudable goal. In addition to some of the other good responses here, my reason for thinking it can’t be solved with better tools alone is that, contrary to what the article suggests, engineering & tools alone are not the primary cause of our complexity, customers and consumers and management are. Business and the need to keep making money is what causes software to need to evolve and change, to support new features that weren’t planned into the architecture, and to develop software in shorter timeframes than is necessary for clean solutions. This is what I’ve seen in a few decades of professional programming, that programmers are perfectly capable of solving hierarchical and cognitively complex problems. Customers or salespeople ask engineering to implement strange inconsistent features, people aren’t happy with your software because your competitor does X, and we’re always walking a balance between time and resources to meet those demands, and there’s never enough of either. A lot of software business, for better or worse, is also driven by fads, so need complexity inducing maintenance just to not feel old and cruddy. The other reason complexity happens is dependencies and DRY thinking, which is often good, but dependencies, centralization of code, and sharing of code are also a risk. Avoiding repetition is what causes abstractions to grow. Using other people’s libraries means that there are complex interactions you don’t understand. Most of the time it’s all fine, but occasionally sometimes it’s not. This trade is made consciously, for the reason that it’s much faster to develop your app using existing libraries, and not reinvent all wheels, and nearly everyone is doing it. The reason that tooling is unlikely to solve this part of the problem is that failures are so often caused by incorrect expectations - someone using a library hoping that it will do X when it only does Y. Even in theory, better tools could only tell us low-level information about what a library does, but I don’t think it’s possible for tools to tell you at a high level what a library can’t do.
notatpeace 4y ago
May be you are looking at a much fine grained composition/decomposition tier to arrive at breaking down "all" software engineering towards simplicity, but anything real world usecase would end up being an aggregate so big, your composition/decomposition tier itself would add to complexity rather than reduce it. BPMN and all of the UI pallet, drag drop coding applications try to do this very thing, this is super successful at smaller scales but breaks the first real world application.
nescioquid 4y ago
I attended a Common Lisp convention at MIT around the year they did away with the SICP course and replaced it with a course using Python to program robots. It came up in the Q&A of one of the talks and the question got kicked to Sussman who was in the audience and gave an explanation. He said that it used to possible, practically speaking, to treat software like discrete components so that you can reason about how to combine the units and predict the result, etc. By contrast, much programming today is bodging together a bunch of libraries whose behaviors are often poorly defined or otherwise opaque so that one often enough has to treat them like black boxes against which you have to apply scientific reasoning to understand how to use and integrate it. Hence Python and robots. Presumably the new course teaches skills around managing this complexity, but the shift in model advocated -- from the metaphor of well-behaved discrete components in composition to opaque units whose inputs and outputs (possibly depending on hidden state) must be discovered was interesting to me, and ultimately adds to the accidental complexity of doing software now. Do you think this sort of problem may have a distinct visual representation that would fundamentally reduce the complexity of library-dominated development? Or are you imagining a whole programming ecosystem built around your idea, so that the fundamental problem Sussmen described is somehow obviated? Or are you really trying to address essential, not accidental complexity?
likortera 4y ago
Don't want to discourage you...but... LOL
deleted 4y ago
[deleted]
eternalban 4y ago
> there are finite and fundamental classes of problems, that underpin the vast majority of problems We already have had, for a long time, a 'universal' model for a specific slice of information processing that consists of a very small set of components: Graphic User Interfaces. One problem is the diversity of domain semantics and all the associated nuances. In GUI programming, this is the tedious bit of naming the components and mapping them to processing elements. Another problem is engineering culture (du jour). A reductionist approach that attempts to leverage structural commonalities requires what is poo poo'd as "boiler plate" and all the associated warts of component oriented programming (including factory-factories). A revisionist shift in mindset is required.
codeulike 4y ago
No Silver Bullet. Frederick P Brooks 1986
dstroot 4y ago
Software complexity in most cases is a direct reflection of business complexity. Changing requirements, overly complex processes, “approval” steps, etc., etc. if you can build it faster and cheaper - great! But you will end up in the same place eventually.
eschneider 4y ago
Nothing on the language side fixes the problem of people just not knowing/understanding the project requirements.
dilyevsky 4y ago
This is like a philosophers stone of swe or what fred brooks referred to as “silver bullet”, good luck ;)
brandall10 4y ago
Sounds like the work of Bret Victor.
compiler-guy 4y ago
His examples are getting a little dated, but reading this, I can’t help but think of Spolskey’s “Architecture Astronauts” essay: “When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there’s a general pattern: sending files. That’s one level of abstraction already. Then they go up one more level: people send files, but web browsers also “send” requests for web pages. And when you think about it, calling a method on an object is like sending a message to an object! It’s the same thing again! Those are all sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it’s getting really vague and nobody really knows what they’re talking about any more. Blah.“ https://www.joelonsoftware.com/2001/04/21/dont-let-architecture-astronauts-scare-you/ https://www.joelonsoftware.com/2001/04/21/dont-let-architect...
- simonh 4y ago
  Came to say the same thing, Joel nailed this 21 years ago. Show me the code, and show me what it can do that other stuff can't.
im_down_w_otp 4y ago
You're describing Mathworks Simulink, more or less. And, I also think it's an incredible shame that it's not a more commonly used tool for forms of software development outside of basically controls engineering.
bmitc 4y ago
While I am a big critic of current software development tools and believe they are nowhere near what they could be, I also believe that there will never be anything that makes software development easy. The reason is that software is a tri-component system: how to do something via computation, how to communicate between people, and how to think about something. We are actually somewhat decent at the first. We are absolutely terrible at the last two. Again, I think there's a lot we can do to improve things, but we will always be pushing against the ceiling of humans to communicate, collaborate, and accurately encode a domain of knowledge. If one thinks of software programs as stored knowledge, then it makes sense why they are so buggy, error prone, etc. It's that the knowledge was never complete to begin with. Throw in the difficulties of computation, and you have the mess that is the software landscape.
seer 4y ago
I definitely share this belief. When I started delving into functional programming I was immediately struck by how whole classes of problems just disappeared once people found some useful abstractions. Programmes that took thousands of lines of code to express could be written in hundreds or even tens. But FP seems a bit stuck nowadays (to my untrained eye) with the breakthroughs from the greybeards of the last decade. Do you have any progress thus far? Anything people can contribute to?
guiriduro 4y ago
Congratulations, you just (re-)invented UML. Or perhaps the "no no, its not UML" semantically anemic C4 thing with lots of blue squares.
mbrodersen 4y ago
The problem with software engineering is not technical complexity. The problem with software engineering is the complexity of humans and the real world. Have you ever worked on software that needs to encode 30+ years of government and employee/union agreements into rules that can be used to automatically decide how much a specific employee needs to be paid overtime in a given month? If you had you would know that no amount of tooling or software will make that problem simple.
agumonkey 4y ago
Out of curiosity, have you watched/read Alan Kay's team work at VPRI ? They managed to make a kind of desktop OS in less than 1MLoC, tangling DSLs to generate the boilerplate from very succint rules (or parsing RFC specs for the network stack). They wanted a 100kLoC but couldn't do it, but I believe it's still doable.

by daenz 4y ago

andrepd 4y ago

Can you elaborate and/or give examples? I read your comment and I can only see vague pseudo-profound statements.

astrange 4y ago

This is essentially Alan Kay/VPRI's old research program. I'm not sure why, but when people get into this they always think dataflow programming will solve everything - we already have Excel and it does do a lot, but it doesn't do everything.

papito 4y ago

It's a solved problem, if you are solving the right one. Most companies out there are solving problems they don't have. The FAANG disciples descended upon the world to preach the gospel of Complexity, and everyone bought it - because everyone wants to be like Google. What, you think "microservices" is a new invention? We called it distributed systems, and we really knew not to go there unless we wanted to decimate our productivity and sleep.

lbriner 4y ago

Good luck but your motivations will probably not line up with the masters who dictate things that become complexity. For example, I look at our software running on .net MVC and think as an Engineer, it simple and knowable but the Front-End Team are less worried about simplicity and more on flashy front-end stuff since that is their job. We end up bolting on a Front-end JS framework and complexity immediately ramps up by like 300%. Are they wrong for wanting a better and more flexible front-end? Not necessarily, I mean all the other companies have cool stuff and if we don't maybe our company dies. Ditto for lots of other examples...

chasd00 4y ago

this reminds me of the Head First series of books. I can still recall specific pages from the Head First Statistics book not because of the content but because every topic (and probably page) had a creative visualization.

rileyphone 4y ago

Me too, and I definitely agree that it is time to move past thinking in "languages" to a more holistic view of programming systems. The fact that as of now, only ~1% of people can access programming and wield the magic of the computer is a tragedy akin to the comparable levels of literacy in the medieval era. At the present, there seems to be a minor renaissance of thought along these lines stemming back to the pioneers of the field (McCarthy, Engelbart, Kay, etc). Everyone has a different take as to what the answer is, but I think "let 1000 flowers bloom" is the best way to find a solution to a complex and largely unknown problem such as software. Personally, I'm afraid that if we fail, the means of programming will become gated behind AIs and society will continue in its trance-like state of passivity instead of creative expression.

notduncansmith 4y ago
We will have to contend with the last bit regardless. Programming requires attention to detail and tolerance for tedium, the erosion of which will only continue as AI grows.
wvenable 4y ago
> The fact that as of now, only ~1% of people can access programming and wield the magic of the computer is a tragedy akin to the comparable levels of literacy in the medieval era. I don't know why programmers think this way. We don't expect everyone to be a doctor, a botanist, a novelist, or a musician but for some reason we think anyone can be a programmer. That's just not the case -- programming is a skill like any other -- it takes some natural inclination, some training, and a lot of practice. Just like any other skill. A programming language is to a programmer what an instrument is to a musician.
- rileyphone 4y ago
  No, but we expect everybody to know reading, writing, and arithmetic. Programming may be more intrinsically difficult than these, but most of the reason it's so hard today is because of solvable problems that beginners run into, to the point of half of expertise is just knowing to avoid all these pitfalls. The downside of the open source developer ecosystem today is that putting it all together is typically a painful process, which is what Replit is aiming to solve. There is a world of difference between a good IDE and a bad one for a beginner, and a lot of that comes down to integration with the language. Anyways, there's a reason programming should be made more widely accessible - it develops thinking skills and rationality when you learn it, as Mindstorms pointed out. Between the anecdotes in the book and things like [0], I think it's a travesty that we aren't pursuing this to its fullest extent, and instead try to teach children Python or JavaScript, two decent languages for software development but not exactly forgiving with beginners. [0] https://medium.com/@stevekrouse/goodbye-seymour-cb712757264f https://medium.com/@stevekrouse/goodbye-seymour-cb712757264f

satyrnein 4y ago

I think I understand what you're getting at, but do you have any examples? Like you said, seeing helps understanding! :-)

deleted 4y ago
[deleted]

AnimalMuppet 4y ago

I don't think you can solve it. (Not a comment on you - I don't think anyone can solve it.) The problem is that software at least to some degree reflects the external environment. Let's say your software is in a financial company. Their software has to enable them to follow all the government financial regulations. Well, the government is following the larger societal "complexify to the point of collapse", and the financial regulations are certainly doing so. That means that the external behavior (the "business logic") of the software is insanely complex. You can't make that go away just by visual programming. But maybe you're not in the financial world. Maybe you're just writing programs for internal corporate processes at some generic company. Well, your software is still subject to the complexity that builds up in the company processes. Again, the programmers can't eliminate that complexity. Or maybe you're writing a customer-facing app - an external-facing web app, or an application that people actually install on their machines. Here you're at the mercy of the product or project manager trying to find new things for the app to do, and they still complexify the app to the point of collapse. The problem isn't that programming is too complicated. The problem is that what we want programs to do is too complicated. Visual programming can't save us from that.

perlgeek 4y ago

I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that convention. Storing both risks the data going out of sync. What's the "right" way to store person names? There doesn't seem to be a simple solution. Another example: we model physical cables (both for power grid and for data transmission) in our CMDB. All works fine, until you suddenly have a Y-shaped cable with three connectors that doesn't fit into your data model. The real world always has these 1% of cases that don't fit the general pattern; if you focus on the 99%, the 1% make trouble. If you focus on modeling every case, you have 10x the complexity, even for the simple case. And then there are things that are moderately complex and security critical, like password recovery workflows. We haven't really found a way to reuse these among different technologies. Like, if you once figured out the perfect password reset workflow with Ruby on Rails, and your next job uses Python + Django, you're back to square one. If somebody has a good idea for how to tackle these problems, please let me know!

nightski 4y ago
As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the same problems is insane. But this is not an easy problem to solve and I don't intend to trivialize it. I think we need to come up with better tools. code sharing through libraries/repositories (like npm) is great, but it can't be the final solution. Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. You could import modules, functions, data structures, anything. But that vision is still a long ways off.
- feoren 4y ago

mamcx 4y ago

I think the same, but that requiere resources. I try to work on the side in a spiritual successor of Foxpro/DBase (https://tablam.org https://tablam.org). I consider a mix of relational/array model fill a lot of ergonomics for this (and it was proved to be right by the family of DBase langs). But what makes this much more complex today is the explosion on OS targets (Windows, Linux, MacOs, Android, iOS, Web), and the requirement to integrate with many other stuff, dealing with many formats (json, xml, ...), is harder to do UIs now and the base support is more inconsistent and mixed as ever... So, is possible to make a simpler tool, I certain of it, but then the developer/user will say "ah, ok, so how this connect to Redis, GraphQL and Amazon Web Services, run this on Android, Windows, parse CSV, ..." and that is what make this very hard at the end...

michaelbuckbee 4y ago

What you're describing is one approach, the other is that you pay a bunch of SaaS/PaaS/IaaS companies to handle a lot of the workload for you so you and a small team can just focus on the business logic of your particular niche.

yhoiseth 4y ago
Agree that this helps in some ways, but it also adds complexity. You can e.g. subscribe to a billing SaaS, but then you have to link your system to the billing system somehow

mathgladiator 4y ago

Welcome to the club! The fundamental problem of simplifying software is humans. Just consider date formats, time zones, and tax codes. Humans love to make complicated things. My philosophy on this has been to take the complex human stuff and stick it in a black box. A professional feather in cap with this approach is called BladeRunner ( https://dl.acm.org/doi/10.1145/3477132.3483572 https://dl.acm.org/doi/10.1145/3477132.3483572 ) which radically simplified the distributed system aspect by putting all the gnarly glue and business logic in a V8 VM (JavaScript). My next thing is a focus on board games where I have invented a programming language and have started to evolve a platform. It's called Adama ( https://www.adama-platform.com/ https://www.adama-platform.com/ ), and I think it is pretty cool. The interesting thing is that the complexity of board games is exceptional. I have a few clues to share. The first is that reactivity, which is found in excel, is a key to simplifying software as this makes the glue more automatic. Another clue is figuring out bidirectional communication which relates to reactivity as a two-way street. However, this is primarily hard because we don't have great things off the shelf to deal with this beyond TCP. For more of a deep dive, check out https://www.adama-platform.com/2021/12/22/woe.html https://www.adama-platform.com/2021/12/22/woe.html which talks about WebSocket. My final clue is that you can't run away from state. So many people offload state because state is hard, and you have to contend with it. I'm building yet another database.

ozim 4y ago
If you look from perspective of software creator that wants to handle all kind of timezones or post codes - yes humans complicate things. If you look from a person or user perspective - time zones or date formats are used in one location or in some context. People living in that context have it a lot easier because most of the time they don't care about other time zones. I would say people simplify things but on local scale. If you want to go global that is your problem not humans that live in one place and use single time zone all their life.

swatcoder 4y ago

This is an admirable and worthy pursuit, but keep in mind that any process you make streamlined: 1. becomes a post from which clever and ambitious people can build complex things. You invent docker to simplify application deployments and then somebody builds a n-dimensional microservice cloud on one side and starts commercializing new hardware architectures on the other. You don't remove the complexity, you just let it move around into new domains. (not a bad thing!) 2. is more temporal than you expect. The "finite and fundamental" problems of yesterday, today, and tomorrow are of different sets -- partly because (1) opens up new problem classes and partly because most problems are inescapably cultural and therefore subject to fashion cycles.

davidgay 4y ago

> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle. So in that view, you can't succeed at making software engineering always simple. Instead, you can make previously intractable problems tractable.

daenz 4y ago
If history is a guide, then you are correct. But I believe complexity is fractal (I think we all know this: "turtles all the way down..."), and that you can structure and manage complexity with recursive rules, so that you only see the resolution that you care about, for the areas that you care about, and everything else is a low frequency representation. Frameworks approximately do this, but for specific domains: they let you organize complexity into well-defined areas and put a pin in them, easing the cognitive load. Then you can handle those abstractions more easily. But yes, to your point, because frameworks typically only tackle a few levels of complexity, you still get that complexity back when you use the framework to advance the problems to the edge of what your framework is designed to address. A recursive/fractal management of complexity will allow all levels of the hierarchy to feel similar, so you are never increasing complexity, only looking at a different resolution. I think the key to this is mapping out the fundamental organizational problems and how they relate to each other at different resolutions.
- ItsMonkk 4y ago
  > tackling more complex problems This is the 'induced demand' argument. As perfection seems impossible, and mistakes inevitable, this seems likely. However, the more I look into issues, the more I realize that a very small number of errors introduced early on is what ultimately causes a plethora of them. Just fixing a very small amount of these mistakes would have untold effects on computing over the long-term. That is, if we can get over the initial switching costs. > small team of engineers It took 2 men to over-take thousands creating Unix over Multix. It took Linus Torvalds - just about alone - a week to create git. We have already seen this prophecy come true. As teams get bigger, communication sales factorially, and the more people you have the more mistakes you make, which increases the size exponentially with each further mistake. What Unix and git showed is that when you put everything into a small team of engineers heads, they can work through the complexity enough until they can do it themselves. > Frameworks One of the things I realized a while ago is that pure/impure and library/framework have a decent mapping between the two. If you have a framework, you give it code and it acts for you, just like impure code. And so the problem we keep running into, is instead of inverting the flow like 'hexagonal architecture' says, we keep piling impure onto impure onto impure. Hexagonal says to not do anything of substance, keep the adapter clean, but each framework sure is doing something. Each layer on the stack we go up, the harder it is to get down. So now we have OS and applications and containers and k8s running micro-services that ends up being run in a web browser, when all we really needed was microkernals.

gtowey 4y ago

> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I think one thing missing from this line of thinking is that these are people. They have their own motivations and egos. They get sick, have families, take vacations, quit. I think part of the way large teams and large companies are structured is a hedge against changing teams with highly variable capacity. It's hard to overstate how much it can hurt a project to lose a someone who has a ton of experience locked up inside their head. The alternative is that I have worked for places that try to treat thier engineers like fungible tokens that can be shuffled and replaced at will. That environment feels extremely dehumanizing and demotivating.

randomsearch 4y ago

I agree with you. I have other things to do, but good luck to you. There’s an awful lot of cultural baggage in coding. Many of the concepts that seem essential - powerful text editors, devils tooling - can be completely removed. It requires rethinking from first principles and being willing to upset the Apple cart, but I believe it is possible.

gjvnq 4y ago

I think that's an honourable but unattainable persuit. A huge part of the problem is the messiness of the real world and costs. While there may be a finite set of fundamental problems, the set of possible hardware and software platforms is ever increasing and each has it's own constraints and strengths. A "universal programming environment" would need something like a "universal hardware interface". That's why things like Java and the Web became so popular despite their poor design. Also, visual programming is a hard thing to make work and human language skills seem to be far greater than human visual skills. Perhaps the best that can be done is "starting from zero" and making sure everything, from the ICs in your hardware to the memory models behind your software is thoroughly tested and formally verified. This is insanely expensive with current technology. Perhaps some fancy math and AI advancements can make formal verification powerful enough for ubiquitous use. Until then, I see little hope for a "universal programming environment".

kansface 4y ago

In addition to the comments below, complexity in software is just as often driven by politics/organizational structure. There is no solution in software to that problem.

diamondap 4y ago
Yes. And that's the root of this problem: > Nobody can subtract from the system; everyone just adds.

jandrewrogers 4y ago

This assumes that complex software problems are decomposable into discrete components defined and analyzable solely by their public interfaces. Complex software frequently cannot be modeled in this way -- it is what makes the software "complex". In reality, components have complex interactions far outside of what is captured in the component interfaces (see also: Hyrum's Law). Surfacing these implicit interactions at the component interface level just creates insanely complex interface definitions, moving the complexity somewhere else. To tie it to something concrete, just about everyone thinks C++ template metaprogramming is unreasonably difficult to reason about even aside from the syntax, but it exists to be extremely powerful at precisely expressing behavioral contracts of a software component that are difficult to do in other languages. Even then, it barely scratches the surface of what would be useful to express for the interaction of those components to be "simple". The number of design parameters of a component that really matter in various contexts are astronomical. No one can deal with reasoning about that many component traits in practice, so the software engineers simply hide most of them -- the complexity is still there and will manifest in unexpected ways because it is not visible at the interface. All systems engineering is complex for this reason. Any non-trivial software system has to be reasoned about as a monolith at some level to correctly address complexity, which creates an unavoidable cognitive load. This isn't a software engineering problem, it is a systems-thinking problem. In chemical engineering, for example, there are often complex system design problems that cannot be adequately addressed by decomposing them into a sequence of sub-problems, all that does is hide major complexity at the interface of the sub-problems.

bigbonch 4y ago
There are many embellished, hand wavy statements in this comment. Could you give real world examples of: * "In reality, components have complex interactions far outside of what is captured in the component interfaces" * How template metaprogramming "precisely express[es] behavioral contracts... that are difficult to do in other languages" I think you contradict your poetry with "Any non-trivial software system has to be reasoned about as a monolith at some level to correctly address complexity". If _any_ non-trivial software system _can_ be reasoned about as a monolith then it is true there exists some model with sufficient simplicity that there are well-defined discrete components.

milkanic 4y ago

Have you looked at Clojure and REPL/data driven development?

meheleventyone 4y ago

Isn’t this the dream since year dot? The problem is the real world is full of leaky abstractions, with different layers evolving at different rates and massive adoption inertia.

nradov 4y ago

I am skeptical of visual programming in general. Some of us just aren't visual thinkers. I find it much easier to conceptualize complex concepts in terms of text and narrative.

tasuki 4y ago
Don't worry - I am a visual thinker and I'm skeptical of visual programming. While it's cute for toy examples, I haven't seen it handle complexity well.

go_elmo 4y ago

I am sure every SE agrees that somehow things are always the same. How about making code pieces algebraic in order to reason and write with them much easier? Like using Haskell?

pebcakID10T 4y ago

I have been developing software for over 20 years and leading teams of developers for the last 10. Please allow me to give you some insight from experience. To your first point, in my experience, the best software, including complex platforms that handle massive amounts of traffic and data, can be built and maintained by small teams. I mean less than 20 developers. The larger the number of developers working on an application or platform has an almost inverse correlation to the speed in which new features are built or bugs fixed. Software architecture has been done visually since perhaps its inception (tools like UML). At most places I've worked, every new project or large feature is diagrammed visually to use as a guide in breaking down the project into component parts. In my experience complexity arises, not from lack of tools or industry knowledge, but from 3 main causes: 1. Inexperienced developer asked to create project, who just starts building without planning beforehand. 2. The main one - business demands features built that were never expected or planned for, and built as fast as possible. This causes developers to take shortcuts, make inelegant and difficult to maintain design choices. And leads to often inscrutable code that becomes technical debt especially after the original developer leaves the company. This will always happen as long as software is used to make a business money. 3. High team turnover - I've seen places where developers came and went so often that there was a myriad of things half started and never finished. How I've solved or helped alleviate these issues: Make the business case to company owners or management that technical debt will be an ever increasing impediment to development velocity and the dev team will need a percentage of work in any given sprint to tackle tech debt issues (as opposed to having everybody work 100% on new features and bug fixes all the time). I have successfully taken a platform that was bug-ridden, difficult to maintain, and where new feature development had slowed to a crawl due to the over-complexity of the software, to a place of stability and ease of development, simply by allowing our team to chip away at tech debt over the course of several years. Tech debt issues were rated by level of complexity, risk to the business in change (regression bugs), and impact on team velocity. We worked on the highest impact, lowest risk items first and kept going until there wasn't much left on that list.

throw827474737 4y ago

How many clever computer scientist (including me ) had this idea and vision at least once? And how many attempts by languages/DSLs/visual languages have already been tried to create? I believe you are wrong, I'd wish you are right ;)

dahart 4y ago

A very worthy, laudable goal. In addition to some of the other good responses here, my reason for thinking it can’t be solved with better tools alone is that, contrary to what the article suggests, engineering & tools alone are not the primary cause of our complexity, customers and consumers and management are. Business and the need to keep making money is what causes software to need to evolve and change, to support new features that weren’t planned into the architecture, and to develop software in shorter timeframes than is necessary for clean solutions. This is what I’ve seen in a few decades of professional programming, that programmers are perfectly capable of solving hierarchical and cognitively complex problems. Customers or salespeople ask engineering to implement strange inconsistent features, people aren’t happy with your software because your competitor does X, and we’re always walking a balance between time and resources to meet those demands, and there’s never enough of either. A lot of software business, for better or worse, is also driven by fads, so need complexity inducing maintenance just to not feel old and cruddy. The other reason complexity happens is dependencies and DRY thinking, which is often good, but dependencies, centralization of code, and sharing of code are also a risk. Avoiding repetition is what causes abstractions to grow. Using other people’s libraries means that there are complex interactions you don’t understand. Most of the time it’s all fine, but occasionally sometimes it’s not. This trade is made consciously, for the reason that it’s much faster to develop your app using existing libraries, and not reinvent all wheels, and nearly everyone is doing it. The reason that tooling is unlikely to solve this part of the problem is that failures are so often caused by incorrect expectations - someone using a library hoping that it will do X when it only does Y. Even in theory, better tools could only tell us low-level information about what a library does, but I don’t think it’s possible for tools to tell you at a high level what a library can’t do.

notatpeace 4y ago

May be you are looking at a much fine grained composition/decomposition tier to arrive at breaking down "all" software engineering towards simplicity, but anything real world usecase would end up being an aggregate so big, your composition/decomposition tier itself would add to complexity rather than reduce it. BPMN and all of the UI pallet, drag drop coding applications try to do this very thing, this is super successful at smaller scales but breaks the first real world application.

nescioquid 4y ago

I attended a Common Lisp convention at MIT around the year they did away with the SICP course and replaced it with a course using Python to program robots. It came up in the Q&A of one of the talks and the question got kicked to Sussman who was in the audience and gave an explanation. He said that it used to possible, practically speaking, to treat software like discrete components so that you can reason about how to combine the units and predict the result, etc. By contrast, much programming today is bodging together a bunch of libraries whose behaviors are often poorly defined or otherwise opaque so that one often enough has to treat them like black boxes against which you have to apply scientific reasoning to understand how to use and integrate it. Hence Python and robots. Presumably the new course teaches skills around managing this complexity, but the shift in model advocated -- from the metaphor of well-behaved discrete components in composition to opaque units whose inputs and outputs (possibly depending on hidden state) must be discovered was interesting to me, and ultimately adds to the accidental complexity of doing software now. Do you think this sort of problem may have a distinct visual representation that would fundamentally reduce the complexity of library-dominated development? Or are you imagining a whole programming ecosystem built around your idea, so that the fundamental problem Sussmen described is somehow obviated? Or are you really trying to address essential, not accidental complexity?

likortera 4y ago

Don't want to discourage you...but... LOL

deleted 4y ago

[deleted]

eternalban 4y ago

> there are finite and fundamental classes of problems, that underpin the vast majority of problems We already have had, for a long time, a 'universal' model for a specific slice of information processing that consists of a very small set of components: Graphic User Interfaces. One problem is the diversity of domain semantics and all the associated nuances. In GUI programming, this is the tedious bit of naming the components and mapping them to processing elements. Another problem is engineering culture (du jour). A reductionist approach that attempts to leverage structural commonalities requires what is poo poo'd as "boiler plate" and all the associated warts of component oriented programming (including factory-factories). A revisionist shift in mindset is required.

codeulike 4y ago

No Silver Bullet. Frederick P Brooks 1986

dstroot 4y ago

Software complexity in most cases is a direct reflection of business complexity. Changing requirements, overly complex processes, “approval” steps, etc., etc. if you can build it faster and cheaper - great! But you will end up in the same place eventually.

eschneider 4y ago

Nothing on the language side fixes the problem of people just not knowing/understanding the project requirements.

dilyevsky 4y ago

This is like a philosophers stone of swe or what fred brooks referred to as “silver bullet”, good luck ;)

brandall10 4y ago

Sounds like the work of Bret Victor.

compiler-guy 4y ago

His examples are getting a little dated, but reading this, I can’t help but think of Spolskey’s “Architecture Astronauts” essay: “When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there’s a general pattern: sending files. That’s one level of abstraction already. Then they go up one more level: people send files, but web browsers also “send” requests for web pages. And when you think about it, calling a method on an object is like sending a message to an object! It’s the same thing again! Those are all sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it’s getting really vague and nobody really knows what they’re talking about any more. Blah.“ https://www.joelonsoftware.com/2001/04/21/dont-let-architecture-astronauts-scare-you/ https://www.joelonsoftware.com/2001/04/21/dont-let-architect...

simonh 4y ago
Came to say the same thing, Joel nailed this 21 years ago. Show me the code, and show me what it can do that other stuff can't.

im_down_w_otp 4y ago

You're describing Mathworks Simulink, more or less. And, I also think it's an incredible shame that it's not a more commonly used tool for forms of software development outside of basically controls engineering.

bmitc 4y ago

While I am a big critic of current software development tools and believe they are nowhere near what they could be, I also believe that there will never be anything that makes software development easy. The reason is that software is a tri-component system: how to do something via computation, how to communicate between people, and how to think about something. We are actually somewhat decent at the first. We are absolutely terrible at the last two. Again, I think there's a lot we can do to improve things, but we will always be pushing against the ceiling of humans to communicate, collaborate, and accurately encode a domain of knowledge. If one thinks of software programs as stored knowledge, then it makes sense why they are so buggy, error prone, etc. It's that the knowledge was never complete to begin with. Throw in the difficulties of computation, and you have the mess that is the software landscape.

seer 4y ago

I definitely share this belief. When I started delving into functional programming I was immediately struck by how whole classes of problems just disappeared once people found some useful abstractions. Programmes that took thousands of lines of code to express could be written in hundreds or even tens. But FP seems a bit stuck nowadays (to my untrained eye) with the breakthroughs from the greybeards of the last decade. Do you have any progress thus far? Anything people can contribute to?

guiriduro 4y ago

Congratulations, you just (re-)invented UML. Or perhaps the "no no, its not UML" semantically anemic C4 thing with lots of blue squares.

mbrodersen 4y ago

The problem with software engineering is not technical complexity. The problem with software engineering is the complexity of humans and the real world. Have you ever worked on software that needs to encode 30+ years of government and employee/union agreements into rules that can be used to automatically decide how much a specific employee needs to be paid overtime in a given month? If you had you would know that no amount of tooling or software will make that problem simple.

agumonkey 4y ago

Out of curiosity, have you watched/read Alan Kay's team work at VPRI ? They managed to make a kind of desktop OS in less than 1MLoC, tangling DSLs to generate the boilerplate from very succint rules (or parsing RFC specs for the network stack). They wanted a 100kLoC but couldn't do it, but I believe it's still doable.