6 ms·
I think part of the problem is that software is hard. I've programmed for two decades. I write non-trivial software modules in a specific domain. It's not as i
by fsloth 6y ago
I think part of the problem is that software is hard.
I've programmed for two decades. I write non-trivial software modules in a specific domain. It's not as if I was averse to analyzing things. And I try. When I can.
Yet. Design is damn hard.
One of the problems is that implementation is often an empirical process of requirements and constraints recovery.
I.e. I cannot create an elegant design before I've implemented anything, because some of the requirements are discovered only when the domain is explored using the code which I write.
One of the problems is that I'm not smart enough to design the system beforehand. And I don't have enough working memory to do so. And I presume many programmers work in similar environments.
I think in the 70's or 80's there was a vivid dream that one day programmers would have advanced CASE tools to help them design better programs. I wonder what happened to that dream. Maybe IDE:s and software systems became so easy to use that everyone prefers to prototype in code and perform empirical constraint discovery just like me...
- ivan_gammel 6y agoThe dream came true. In a modern non-software startup environment people often write code only as a last resort. I regularly see BizDev people setting up new business process automations with services like Zapier, Jira, Zoho, AutoPilot etc. There are plenty of CMS systems, customizable BPM, CRM, ERP etc where you can define UI and logic without writing a line of code and it just works. The cost reduction compared to the traditional coding is tremendous and enables small IT teams to run quite sophisticated businesses. Developers usually don’t see this, locked within their code-based programming paradigm, but this is already real.
- fsloth 6y agoThanks for this! You are quite right. For the space that is dominated by "CRUD" apps there are quite a lot of alternative implementation pathways than pure coding. And as most of the business value lives between the UI and the database, I understand why there is no driving incentive to create similar high quality tools for numerical, data intensive programming tasks that run on the customers desktop. We work in a niche, hence must be self sufficient.
- wiz21c 6y agoIMHO, the business value lies in the data and the way they are structured. The user interface is important to the users, but the data are what make the business run longer than its users. Basically, without the data you can't provide the service; or worse, with bad quality data, you can't provide a good service. I've spent (in bizdev, e-gov) tons of time figuring out badly structured data, fixing data quality issues, handling tons of administrative tasks to access sensitive data, etc. I think it's up to a point where the structure of the databases actually influence the structure of the government. That's because nowadays, the databases and the business rules they embody are so complex that even the civil servants tend to forget how they work.
- FeepingCreature 6y agoSounds like a version of Conway's Law: "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure." The assumption was that the communications structure would come first and then the design would imitate it, but with really ossified hard-to-change systems it seems plausible that the departmental organization would shape around it.
- TeMPOraL 6y ago> but the data are what make the business run longer than its users. How often is that the case these days, though? Currently, when I subscribe to a service or buy a product tied to a service, I feel there's a 50/50 chance the company will shut down the service (or itself) while I'm still using it.
- zelphirkalt 6y agoLet me tell you, mostly these solutions suck. There are several reasons for this: (1) leaky abstraction, (2) not the correct abstraction, which works in all envisioned cases without limiting you, (3) often you do not want to only use an API, but instead change a tiny thing and that needs code or accept limitations (4) the existing API is weird or has a weirdly structured result (5) it is inefficient (6) you depend on a third party (7) you might have some rate limits, and probably more reasons. If you got capable developers, let them do their job and create a flexible (expecting changes, simple design, usually not an elaborate many classes design) solution, that actually fits your situation. Developers are (should be) abstractions specialists. They should be able to find working ones in most cases. Or live with limitations and the spread of thinking, that some things are not possible in various appartments. Mental barriers, that stop creative thinking, simply, because the tools do not exist.
- arethuza 6y ago"mostly these solutions suck" I think that's a bit harsh - a lot of these tools can be pretty powerful if they are used for the kinds of tasks that they are well suited for. However, stray into a task that is just that bit more complex than what they are good for and they can be an utter nightmare. Of course, deciding whether a particular task is a good fit for a particular tool can itself be a tricky question.
- ivan_gammel 6y agoWell, as a software engineer who wrote first line of code in 1991 and as a CTO of a medium sized company I would disagree here. These solutions do not suck. They work, they are cheap and they are efficient in addressing the business needs. You don’t always need a developer for simple automation, just like you don’t need a computer to calculate 2+2. Abstraction leaks, third party solutions and other things you mention are not the problems that always need a solution. Developers often overthink scalability and invest time in perfection of a code which will become obsolete in a year or two because of the change in the business model. It is there you can find the art of engineering: in finding the right problem to solve, rather than working on the problem for which you know a conventional solution. Zapier is unconventional, but it takes 30 minutes there for something that a development team would require a week or two. The limitations do exist, but it often takes some courage to understand that in your specific case you can ignore them.
- collyw 6y agoIt was excel a decade ago,and those only work well up to a point.
- ivan_gammel 6y agoIt’s still there together with the Google Spreadsheets. And that point is quite high :)
- collyw 6y agoI guess viewing the point as quite high is pretty subjective. Plenty of important business practices rely on them, so in that sense you are correct.
- systemvoltage 6y agoYou've nailed it - I am an inexperienced software engineer but when I lay down the architecture of a particular problem, think about it in excrutiating detail upfront; nothing goes according to the plan. Reality hits you like a brick and new things emerge when actually writing code. Even ambiguity in requirements surface when actually implementing the logic in code. So no matter how good the requirements are, theyll definitely need additional refinement. Software Engineering is really hard I've recently discovered and I value your decades long perspective.
- wslack 6y ago100% - and even the best developer may not understand all of a user's needs until they get their hands on a prototype. We should always expect to learn and pivot as we learn more about users.
- Xixi 6y agoI would even rephrase it that way: "even the best user may not understand all of their needs until they get their hands on a prototype."
- adwn 6y agoExactly. I've never seen a (non-trivial) requirements specification that doesn't contain holes, contradictions, imprecision, or requirements that will be dropped or altered during development.
- rimliu 6y agoEverybody has a plan until they get punched in the mouth… - Mike Tyson
- leafboi 6y agoI've found that FP using function composition or other higher order forms of function like composition mitigates the design problem that you and many others complain about. Essentially in FP you design your entire program as a single immutable function expression that's made out of a pyramid of layers that's constructed of smaller closed lego like functions composed together. People in the FP world call it the point free style. The type signatures guide the architecture of your program and can easily be replaced decomposed or recomposed into higher or lower forms of abstraction without compromising your overall program. Design comes naturally like building something out of legos... Easily configurable, and minimal planning or need to hold the entire program in your head. If the design is wrong like legos your program is easily reconfigurable. The main factor that allows lego like design is immutability. A lego that mutates state outside of its own scope is a lego piece that cannot be modular. FP by making everything immutable makes your program legos all the way down meaning your smallest primitive can likely be broken down further to reuse in other places and recomposed to form other higher order abstractions that are easier to reason about. In both OOP and procedural programming your designs can never have this level of flexibility due in to modules being tied together by shared mutable state. Of course there are costs to immutability: both low level costs such as memory management and high level costs such as graph algorithms that require mutability to be effective. There are ways to deal with these issues but they are not trivial. In the end all FP programs must "cheat" in some way and have a small portion of their program actually mutate something. Overall though, in all my years of programming I have found FP to be the best overall answer to the design problems that most programmers run into. It's by no means a perfect answer but it is the best I've seen and unfortunately not applicable to all domains.
- fsloth 6y agoAgree FP is great as you exposed for a large category of tasks. Agree it is not suitable for some fields, and hence, not a panacea. FP offers a better syntax for most of the programs I write, hence it makes the initial effort easier. Especially "compiler like" or "interpreter like" programs more often than not just fall into a pit of success. But still - my original point remains. I am unable to design a program beforehand. The iteration is just faster with FP. On one point I disagree: " In both OOP and procedural programming your designs can never have this level of flexibility due in to modules being tied together by shared mutable state." There is nothing that stops one from writing most OO or procedural programs using immutable style. In fact, this style makes those programs better as well. Just try it. Never mutate an object. Each method, if they must mutate something, then let them return a new object instead. This style is not applicable everywhere, and due to lack of garbage collected immutable datastructures in the C++ basic library, for example, the area where immutable style can be used is smaller than in a FP language. And you can't use it in those cases naturally where you actually need to mutate existing storage. But, say, an array of 10 elements? Immutable! Most strings smaller than n * 1000 chars? Immutable! Etc.
- mesid 6y agoHow does this compare to non-software industries? Such as construction.