14 ms·

Software disenchantment

• hyperpallium 8y ago
  People won't pay for efficiency. People buy solutions to their problems (features), not efficiency. If it needs to be efficient, it will be (the games example). The death of Moore's Law may rejuvenate efficient software. BTW If you use old unix tools on a phone, it's super fast (in a terminal emulator like http://termux.com http://termux.com). Sadly, the idea of abstractions enabling you to think better (like Alan Kay's point of view is worth 80 IQ points and maybe Jobs' bicycle for the mind) doesn't seem to work, at least for software abstractions. It's far more work than can be justified - more appropriate for pure research (which is instead preoccupied things like esoteric type systems with nice mathematical properties).
  • pjmlp 8y ago
    > If you use old unix tools on a phone, it's super fast Of course it is fast, software designed for a PDP-11 running on a pocket Cray.
  • throwaway080383 8y ago
    Just a few months ago, a very senior engineer on my team made this same observation about Moore's Law after I lamented that bit twiddling tricks and knowledge of obscure algorithmic speedups don't often end up being useful in practice.
  • class4behavior 8y ago
    >If it needs to be efficient, it will be (the games example). Yeeah, no. Definitely a no. >People won't pay for efficiency. People buy solutions to their problems (features), not efficiency. Not wrong. Businesses pay for efficiency.
  • xchaotic 8y ago
    The only sane reason that AirBnb or Uber are worth as much is that they promise to be more efficient than traditional companies. SO the 'market' pays for efficiently and a lot of digital revolution has been all about making existing processes slightly more efficient (Amazon online shopping experience vs physically going to the different shops etc) So the market definitely value the efficiency where the cost of developing that efficiency does not outweigh the benefits. Typing speed and cursor refresh are not limiting factors for programmers so there's diminishing return in making text editors refresh >120hz. Simple business logic and OP can lament as much as he wants to but we have limited resources, mostly time and that time is usually better spent writing business logic than optimising text editor for the 100th time.
• udba 8y ago
  I agree with the author. Often I find it refreshing to step away from the absurdity of software development and go work on my car. Working on your car exposes you to the fruits of a bona fide engineering process: objects that are carefully designed, work reliably, and are made with serviceability in mind. They aren't just slapped together and thrown out the door like (100-epsilon)% of software out there.
  • jaclaz 8y ago
    I see you have an oldish car ...
    • pknopf 8y ago
      But it doesn't break down and require constant maintenance as the parts it uses become obsolete and need constant updating. I'm just trying to get from A to B, not fly to Mars.
  • expertentipp 8y ago
    I would prefer not to touch the turbines attached to downsized 1L 120HP engine, software update only at authorized shops cost north of 150 EUR, the entertainment system had access to youtube initially but was cut off after 12 months, the LED system of headlights was likely someone’s PhD dissertation. Car industry is the eqivalent of web front-end in the software world, except one is unable to just shut down the browser.
    • udba 8y ago
      I agree with your sentiment, the car industry in its current state is getting pretty ugly. I choose to avoid these trends entirely, and I'm happy to keep driving my 2.4L naturally aspirated, 5 speed manual, touchscreen-less shitbox until it dies. Parts are still cheap and it's easy to work on.
• alcio 8y ago
  > Recently our industry’s lack of care for efficiency, simplicity, and excellence started really getting to me, to the point of me getting depressed by my own career and the IT in general. Loading this website resulted in 5.3MB being downloaded over 42 requests.
  • hadrien01 8y ago
    I have only 2.8MB on my computer with the adblocker (the website has Google Analytics and a Twitter widget), mostly images. Fonts/Text/CSS: 120KB. And the website works perfectly without Javascript.
  • oftenwrong 8y ago
    I block everything but HTML by default. One request, 10.49 KB transferred.
    • wingtangwong 8y ago
      +1 much of the weight/bloat of web pages are: javascript code, massive amounts of media, ADVERTISEMENTS!!!!, etc. The base html is fairly lightweight. ;)
    • phyzome 8y ago
      With images, it's 14 requests, 24 KB, a little over a second. (I block JS.)
  • paulddraper 8y ago
    I don't think he authored that though. He didn't say he didn't use software; he wishes they were lighter.
• vbsteven 8y ago
  I feel the same way as the author. I have a theory that we're not caring as much about performance and good software because we don't have to, because Moore's Law has led to machines that are fast enough to run our crappy and hastily written code with acceptable performance so the majority of users do not really care about it running more efficient. In the past, when computing resources were constrained we had to care about performance just so we could release software that worked good enough so people actually wanted to use it. How do we get people to care about performance and efficiency in software? I don't know.
  • felix_nagaand 8y ago
    When the cost of increasing performance enough to satisfactorily run spaghetti is greater than the cost of writing efficient code.
  • xg15 8y ago
    > I have a theory that we're not caring as much about performance and good software because we don't have to, because Moore's Law has led to machines that are fast enough to run our crappy and hastily written code with acceptable performance so the majority of users do not really care about it running more efficient. Depends what you define as "acceptable" and as "users caring". The bloat does have actual consequences, like slow websites and greatly inflated bandwidth costs (which has direct consequences for users on metered connections). I've heard complaints from completely non-technical friends how frustrating it is to work with slow, unstable software. But of course they can't identify more detailed causes because they don't know the technical details (or are interested in them). Likewise, they can't stop using the software because they need it to do their job. So I think there is not much else they could do to indicate they "really cared".
    • commandlinefan 8y ago
      I spend a significant portion of my work-week defending my employment - inputting time cards, updating JIRA tickets with "time spent", attending status meetings... all in the name of "efficiency". And on some level, I understand - nobody wants a Dilbert-esque Wally type wasting space. Yet the corporate attempts at maximum efficiency fail more horrifically than you would be able to imagine anything failing. Not only does the time spent proving that you weren't wasting time waste time itself, it's also so trivially gameable that the people who thrive are the ones who are MOST effective at being silently inefficient.
• almostarockstar 8y ago
  It's an interesting game we are playing. Take websites as an example. We have to build websites using CSS and HTML and JS because that is what works in browsers. But browsers have to build DOMs, layout engines and JS engines this way because that's the way we write code. It's nobody's fault. It's evolution with a bad fitness function. We need a new start. A new paradigm.
  • xg15 8y ago
    No, there were some deliberate design decisions. E.g., there was an attempt at a new start: XHTML 2. This got boycotted by browser vendors out of a mix of politics and business priorities. Instead they went with HTML 5, which generally sees scripting as the core to make non-trivial web applications and also explicitly encourages JS libraries and frameworks. So I don't think it's surprising that today, pages are piles of scripts on top of scripts.
  • metalrain 8y ago
    In a sense AMP is the answer, produce less by having explicit limits while not throwing all tooling out. While AMP is somewhat more technically complex it's much simpler when you consider what it disallows. However technology/solution is only tool, real problem is in the domain of users, how to make useful systems, how to make users care, technology is irrelevant for most of us.
    • emasser 8y ago
      So in a sense, web browsers are a bit to promiscuous. Website builders would have more incentive to properly engineer their websites if browsers were pickier. One issue with that is that it is the websites visitors who are most likely to see the errors and not the designer of the site.
• c64e7a53fa6d976 8y ago
  Just give up on modern software and join the demoscene.
  • MaxBarraclough 8y ago
    Does it pay the bills?
• sebastianconcpt 8y ago
  To capitalize current programmers know how with those expectations of performance would require a radical shift in hardware. It would demand hardware that is optimally designed for multi-paradigm high level languages. Ironically, that was what the Burroughs 5000 was: https://www.smecc.org/The%20Architecture%20%20of%20the%20Burroughs%20B-5000.htm https://www.smecc.org/The%20Architecture%20%20of%20the%20Bur... "The Burroughs B5000 was designed from the start to support high level languages (HLLs), specifically ALGOL and COBOL. The initial features supporting this were the hardware-managed stack, the descriptor, and segmentation. A redesign of this system, the B6000, extended the concept by moving the tag bits to their own field, external to the 48-bit data word. The instruction set was also revised and extended, and has remained essentially unchanged ever since. HLLs are supported by hardware that allows virtual memory, multiprocessing, fast context switching and procedure invocation, memory protection, and code sharing." Emulator project: http://retro-b5500.blogspot.com http://retro-b5500.blogspot.com
  • pknopf 8y ago
    People are focusing too much on the "performance" aspect of the article. The "Programming is the same mess" section is what really resonates with me, specifically this: > And dependencies? People easily add overengineered “full package solutions” to solve the simplest problems without considering their costs. And those dependencies bring other dependencies. You end up with a tree that is something in between of horror story (OMG so big and full of conflicts) and comedy (there’s no reason we include these, yet here they are):
• voidmain 8y ago
  One day through the primeval wood A calf walked home as good calves should... https://m.poets.org/poetsorg/poem/calf-path https://m.poets.org/poetsorg/poem/calf-path
  • dfee 8y ago
    Fantastic
  • Sharlin 8y ago
    Path dependence, literally. https://en.wikipedia.org/wiki/Path_dependence https://en.wikipedia.org/wiki/Path_dependence
• mpolun 8y ago
  Software hasn't been getting worse -- it always sucked. It used to be fast and small because there was no other option. Also: > @jckarter: A DOS program can be made to run unmodified on > pretty much any computer made since the 80s. A JavaScript > app might break with tomorrow’s Chrome update LOL, DOS programs were crazy nonportable. They usually can only be run in VMs these days. Comparing it favorably to the most stable and portable API (the js ecosystem is another story, but the browser APIS are super-stable and never break) in the history of software is hilarious. I mean, all the flaws of DOS programs are totally reasonable given their constraints, but the comparison gets this totally backwards.
• tabtab 8y ago
  I agree that something is out of whack. In-house plane-jane CRUD development used to be pretty simple and quick in the 90's. One could focus on the domain analysis side instead of micromanage tech. The IDE products had glitches, but got better every release, including deployment. The Web bleeped it all to heck and back, and nobody seems interested in promoting the standards to fix it. We toss in JavaScript gizmos to attempt to improve the UI to desktop standards, but these gizmos are clunky and browser-version-sensitive. Maybe the desktop era tools were artistically ugly, but much easier to develop and maintain. PHB's seem more swayed by UI eye-candy than practicality. It's great job security, but hugely wasteful. Our industry needs some deep soul-searching.
  • rwcorbett 8y ago
    > We toss in JavaScript gizmos to attempt to improve the UI to desktop standards, but these gizmos are clunky and browser-version-sensitive. Agreed! doesn't anyone know how to write vanilla JavaScript anymore? relevant: http://vanilla-js.com/ http://vanilla-js.com/
    • BigJono 8y ago
      Pretty hard to learn vanilla JavaScript when you're stuck in the mud 8 abstractions above it, trying to keep up.
  • Sharlin 8y ago
    I think this is what pisses me off the most. The frontend is a complete clusterfuck, but the serverside is almost as bad. Everything is just half-assed and you end up spending most of your time fighting the tech. The worst thing is people who have drunk the Kool-Aid and just stare you, uncomprehending, when you complain that lunatics are running the asylum and the only way to win is to not play. Or at best agree but have ended up silently accepting the situation as a coping mechanism of some sort, like people in abusive relationships. "It's not that bad, there are good moments too, and besides, I'd have nowhere to go anyway." I think I'm done with the tech industry. 98% of the sort of programming people actually pay you for is about as pleasant as a root canal. I'm currently on a sick leave and seriously entertaining the idea of changing careers to something entirely different. Like gardening.
• T3RMINATED 8y ago
  tonsky. the only way to get out is in a body bag. once you are in, you are in for life.
• gpsx 8y ago
  Software is hard. I blame it in Moore's law, in the sense that computers have gotten more capable but we programmers have not. Its like the brain can only hold so much before it gets overrun, and we reached that point several years ago. Working with large programs is very difficult, even for the best architected ones. And of course we can't expect all software to be the best software. The result of these big programs is people blindly using modules created by others, built on top of ever more modules. In theory this is good though. It is specialization. But we haven't done a good job of it. I think if we want to make software easier to write (and as a result better) the place to start is on the process of programming with more specialized pieces, be it a better module system or domain specific features in the programming language. I think it comes down to managing the complexity some a person can better view the big picture.
• jasonhanley 8y ago
  If anything, the rants about this are getting better. That's probably a good sign. This rant is at least 10x better than mine from last year :) https://www.jasonhanley.com/blog/2017/08/when-did-software-go-off-the-rails/ https://www.jasonhanley.com/blog/2017/08/when-did-software-g... To the author: I'm with you. Recently rebuilt my website and blog as an experiment in efficiency.
• emasser 8y ago
  Well, if you take the analogy of the car, that today is working quite efficient and hassle free, that, I would argue, is in large part due to regulation and bureaucracy. Something that the Internet and computing does not have at the moment. That word has some negative connotations, but I do not think that is fair, considering the cars, or the airplanes, history. We assume that the car we purchase has seat belts and catalytic converters which will prevent lots of injuries and effects on the climate and living things. These are mandated by laws in many places, and that is a good thing, because we can worry about the entertainment system, service deals, colour and style and still get a car that meets certain "basic" requirements. I say "basic" as they can be quite advanced, we would hardly buy a car without ABS or a crash rating today. Computer programs and websites do not have to meet any regulations and they therefore easily succumb to bloat, because there is no one being blamed for them being unusable, which is broad, but a website over, say, 1MB is quite large. Plenty are wasting CPU cycles to render and compute, which might not be as polluting as exhaust from a car, but it is polluting and stealing time. Early cars had to change oil very often or have it refilled, because it leaked in to the combustion chamber or gasses leaked out. This causes emissions to be quite hostile, which is bad because it affects the environment and living beings respiratory systems, among other things. We then got laws regulating this and manufacturers were keen to improve the performance of the engines. Today there are 1,6 liter, four cylinder engines that work well pulling loads and still manage to have a good fuel economy when driven under average conditions. Another thing is that customers got fed up with steel wheels and rubber tires that blew out all the time and the wheels on our transports improved as well, but that took some years to happen, the Internet and personal computers are still quite young and immature, there is still time for it all to mature quite a bit, and this article is a good step on the way towards that in my opinion. UX is maybe not as simple to regulate as cars and their emissions and safety standards but I would say that we should not be all to opposed to some more requirements or maybe even laws regarding how we operate computers and programs in some respects and let the market, users that is, dictate what is desired in other areas. We might like to think that the internet is the wild west, so to say, but this has led to some mighty bloat and headaches for all involved, as this article shines a light on.
  • thrower123 8y ago
    And much like software, there are many of us who get nostalgic about the days when automobiles just worked, and were knowable, not inscrutable piles of nonsense.
• svilen_dobrev 8y ago
  yeah, wellcome to the club. i am making software for 30 years, and... IMHO, although last 10 years might have been worst, in terms of bloat and inefficiency and dont-cares-ness, it's not just that. That's a result.. It's more like IT industry as such is trying to grow for the sake of growing. At any price. Much like a lot of other industries and/or institutions, trying to exist for the sake of existing. And they will.. as long as there are users of.. "Each train has its passengers, the saying goes."
  • paulddraper 8y ago
    > Much like a lot of other industries and/or institutions, trying to exist for the sake of existing. You've just described the purpose of literally every life form.
• floatboth 8y ago
  > Windows 10 takes 30 minutes to update. What could it possibly be doing for that long? Backing up the old Windows folder, decompressing (I guess downloadable updates use something very efficient in terms of compression like lzma), and probably some sleep() — I guess the update being slower makes users feel like it's something substantial. > Android system with no apps takes almost 6 Gb That "System" figure is the size of all partitions that aren't data. Quite a lot of that is literally wasted space.
• alexeiz 8y ago
  Nice try. It's everyone's wish that somebody makes efficient software for them. But when it comes to themselves, they just lump some random components together that seem to do the job and call it a day.
• renox 8y ago
  1) some of these complaints are invalid: yes Linux kill a "random" process when there is no more memory available, but except in 'static' configuration where everything has a known size, what's the alternative? You can have a misbehaving application use lots of memory but it's a normal application whose memory allocation fail, what do you do? 2) this bloat isn't new: I remember being amazed by BeOS responsiveness compared to Windows or Linux but now I spend 80% of my time in a webbrowser and I doubt WebKit is going to be faster on Haiku than it is on other OSs..
  • slededit 8y ago
    1) Don’t promise memory you don’t have. On windows VirtualAlloc will fail if it doesn’t have enough page file space to satisfy the request. This allows the application to do something smarter than with the false promise it gets from mmap. 2) Really depends. Applications spend a lot of time in syscalls.
    • Piskvorrr 8y ago
      Yup, pretty sure that "overcommit" should never have been the default option (although it is quite easy to disable, comparatively few people know that it even happens).
    • renox 8y ago
      1) it doesn't really solve the issue except if you can allocate everything you need at startup: if you need more memory but someone else had a bug which used all the memory you also have a problem..
      • slededit 8y ago
        You may have caches or other optional buffers you can free. Or you may choose to save or do some other action to save user data. It doesn't solve the fact you don't have memory, but it does let you at least do something about it.
• paulddraper 8y ago
  I share the sentiment, but I also appreciate the truth. > Text editors! What can be simpler? On each keystroke, all you have to do is update tiny rectangular region and modern text editors can’t do that in 16ms. I use text editors with syntax highlighting. Also IDEs with completion, error highlighting, etc. Assuming we are talking about those, updating a rectangular region is not "all you have to do". (Though it is the only thing that has to be done synchronously.) > Windows 10 takes 30 minutes to update. What could it possibly be doing for that long? That much time is enough to fully format my SSD drive, download a fresh build and install it like 5 times in a row. You can format an SSD, download a multi-gig image and install it in under 6 minutes? > The whole webpage/SQL database architecture is built on a premise (hope, even) that nobody will touch your data while you look at the rendered webpage. What? > Ever seen this dialogue “which version to keep?” I mean, bar today is so low that your users would be happy to at least have a window like that. I'm failing to see the problem with that dialog. It's hardly new. Vim: WARNING: The file has been changed since reading!!! Do you really want to write to it (y/n)? > Nobody thinks compiler that works minutes or even hours is a problem. Everyone thinks that a problem. It's the reason that memory-bloating scripting languages the author is shitting over have become so popular. > We put virtual machines inside Linux, and then we put Docker inside virtual machines, simply because nobody was able to clean up the mess that most programs, languages and their environment produce. Huh? When will you put virtual machines inside Linux and then Docker inside those virtual machines. (Maybe a Docker dev testing against different kernel versions?) > Linux kills random processes by design. It can kill processes (configurable). But AFAIK it doesn't do it randomly. > Programs can’t work for years without reboots anymore. Anymore? Really? I reboot my Windows computer one-tenth as often as twenty years ago. --- Well, there might be problems, but as least the author is not overgeneralizing, right? > Nobody understands anything at this point. Neither they want to. > What’s worse, nobody has time to stop and figure out what happened. > Nothing stops us from making build process reliable, predictable and 100% reproducible. Just nobody thinks its important. > We’re stuck in local optima and nobody wants to move out.
  • tonsky 8y ago
    > You can format an SSD, download a multi-gig image and install it in under 6 minutes? 5 minutes to download 4 Gb at 100 Mbit/s, 0,5 min to write 4 Gb at 1Gbps > I'm failing to see the problem with that dialog. It's hardly new. It’s always been a problem. Trouble is, you want work from both versions combined, not to select one and destroy another > Everyone thinks that a problem. Yet new languages (e.g. Rust) are still developed with terribly slow compilers > It can kill processes (configurable). But AFAIK it doesn't do it randomly. Well, I meant randomly in a sense that you can hardly predict or control what will happen. Of course it’s a determenistic algorithm, but engineers treat it as a black box. > Anymore? Really? I reboot my Windows computer one-tenth as often as twenty years ago. Why do you need to reboot at all?
• agumonkey 8y ago
  an IRCer reminded me of http://harmful.cat-v.org/software/ http://harmful.cat-v.org/software/ ;)
• AManWithNoName 8y ago
  Can't agree more. In the early days IT was a science. Now it's a playground.
• jodrellblank 8y ago
  I think I first read this rant in a 1997 article by Dr Peter Cochrane, former Head of Research and Chief Technologist of British Telecom [1]: "In the past few years I have watched generations of word processors, spreadsheets, and graphics packs transcend the useful and user friendly, to become fiendishly complex - from auto-spelling checks that irritatingly pop on to the screen as you type, to the graphics-by-questionnaire that realise the wrong format in five easy stages. Application changes of this type beggar belief. Not only do they consume vast amounts of storage, they reconfigure commands, change names and locations, present a vast range of never to be used (or discovered) options that just confuse users. [..] It can only be that commercial considerations prevent us having a cut down and basic set of applications that are backward, forward and sideways compatible. Writing a letter, book, business case or report does not demand the capabilities of an entire publishing industry." - "How to upgrade your stress" - http://archive.cochrane.org.uk/opinion/archive/telegraph/1997/12-8-97.php http://archive.cochrane.org.uk/opinion/archive/telegraph/199... That was written on a 36Mhz laptop with 20MB RAM and 0.5GB disk space. Within weeks, he wrote "My recent computer hardware upgrade, and simultaneous software downgrade, has resulted in huge performance improvements. More or less all delays between hand, screen, and eye have been removed. Applications open instantaneously once booted up and files save in a second." - in "Beyond Biology" - http://archive.cochrane.org.uk/opinion/archive/telegraph/1997/18-11-97.php http://archive.cochrane.org.uk/opinion/archive/telegraph/199... And then he goes on to tear apart tonsky's argument about aircraft and cars: "If we could manipulate the space in materials we could perhaps reduce the weight of an aircraft by 90%, suggests Peter Cochrane [..]" - "There's nothing to the Universe" - http://archive.cochrane.org.uk/opinion/archive/telegraph/1997/11-11-97.php http://archive.cochrane.org.uk/opinion/archive/telegraph/199... [1] http://www.cochrane.org.uk/bio/ http://www.cochrane.org.uk/bio/
• javierj 8y ago
  I think sometimes we, as programmers, are too worried about our own experience coding than about the final product and its quality. And some of that might have been caused by terrible deadlines imposed by PjMs or "Architechts".
• bcheung 8y ago
  Love this article. I gave an internal talk recently that was about the sad state of programming right now. How we used to set a variable in assembly language: mov [foo], ax How we do it in React / Redux // constants.js export const SET_FOO = 'SET_FOO' // actions.js import { SET_FOO } from './constants' export const setFoo = foo => ({ type: SET_FOO, payload: foo }) // reducer.js import { SET_FOO } from './constants' const initialState = { foo: 0 } const reducer = (state = initialState, action) => { const { type, payload } = action switch (type) { case SET_FOO: return { ...state, foo: payload } default: return state } export default reducer // Component.js import { connect } from 'react-redux' import { setFoo } from './actions' class Component extends React.Component { handleSetFoo = foo => { this.props.dispatch(setFoo(foo)) } } export default connect()(Component) I don't understand how people don't see something is horribly wrong.
  • commandlinefan 8y ago
    Don't forget Enterprise Java: public class VariableSetter { private String variableName; private String variableValue; private VariableSetter(); public VariableSetter variableSetterInstance() { return new VariableSetter(); } public void setVariableName(String variableName) { this.variableName = variableName; } public void setVariableValue(String variableValue) { this.variableValue = variableValue; } public String getVariableName() { return this.variableName; } public String getVariableValue() { return this.variableValue; } } public class VariableSetterFactory { private VariableSetterFactor() { } public static getVariableSetterFactory() { return new VariableSetterFactory(); } public VariableValueSetter getVariableSetter() { return VariableSetter.variableSetterInstance(); } }
  • sha90 8y ago
    This is extremely hyperbolic, and I'm sure you know this. Here's how you set a variable in React: const foo = 1; What you're doing in your complicated sample is not _just_ setting a variable, you're also exposing it to a KVO subscription system that you could never represent succinctly in ASM. What's sad is how programmers communicate programming concepts right now, with quick digs and hot takes and zero actual critical thought to what is being compared. I'm disappointed that you're spreading FUD in your internal talks.
• deleted 8y ago
  [deleted]
• OmarGonzalez 8y ago
  I think this is a symptom that, as engineers we are now relegated to a second plane, our focus is just to accomplish the next task set by people who doesn't care or understand a bit about software engineering. You can read it here multiple times "we are pay to make features", the not so passive sentiment that we should be relegated to code monkeys that accomplish business objectives is just so sad to me.
• Sohcahtoa82 8y ago
  I put some of the blame on an unhealthy insistence of code reuseability and sticking to paradigms. The result is so much code that is just abstractions piled onto abstractions piled on to more abstractions until you get the famous FizzBuzz Enterprise Edition [0] that I'm sure we've all seen by now. Nobody wants to just write some god damn code anymore. They include entire frameworks instead of just writing a couple helper functions. [0] https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpriseEdition https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
  • winter_blue 8y ago
    > include entire frameworks instead of just writing a couple helper functions > unhealthy insistence of code reuseability The problem you're alluding is that until recently, there was very poor support on the web for the equivalent of Unix strip[1] or Proguard[2] (often used by Android devs). We need reliable and accurate dependency analysis to be able to properly eliminate unused code. Today, with ES6 modules and a recent version of webpack, this is partly possible. Unused modules are not included in the final output file. I noticed a drastic drop in the final minified JS file size, after moving to a configuration that enabled this sort of (module) stripping. [1] https://en.wikipedia.org/wiki/Strip_(Unix) https://en.wikipedia.org/wiki/Strip_(Unix) [2] https://stuff.mit.edu/afs/sipb/project/android/sdk/android-sdk-linux/tools/proguard/docs/index.html#manual/introduction.html https://stuff.mit.edu/afs/sipb/project/android/sdk/android-s...
    • Sohcahtoa82 8y ago
      Those shouldn't even be needed. To give an example, let's say you need to calculate SHA-256 hashes in your program. People would rather include the OpenSSL library than just add a SHA-256 function to their code. Instead of a single function (that maybe has a couple helper functions), they add a massive library full of features they don't need, taking up memory, bloating the executable size, and increasing load time. If you're using less than 10% of a module, you should be questioning if you even need that module at all.
      • 8y ago
• jonathanpowers 8y ago
  Great article! I've been thinking about this problem since 2006, and there are probably a hundred reasons why things are as bad as they are. Many commenters here have hit upon some: misuse of abstractions (they should be used to save time, NOT save knowledge), insistence on code reuse, etc. To add a few more: treating programming like a form of manufacturing instead of an art, believing that the latest language/tool/process is a silver bullet (and even believing that it's new!), not realizing/accepting that constant refactoring is vital, worshiping teams at the cost of individuals, not letting teams form themselves, and a huge one is this: trying to eliminate problems and/or complexity instead of mitigating them with tools and skill-building. If you try to eliminate a problem or complexity, you will succeed, but at the cost of creating at least as many other different problems than you had before, and you'll just end up moving the complexity to another place. Later, when you realize you still have problems, you'll try the eliminate them too, and do the same thing all over again. Building layer after layer, always increasing your problems and complexity while attempting to reduce them. (This is exactly the same as government controls breading more controls, and it happens for the same reasons.) When I first started out, if you were programming computers at all (microcomputers, anyway) you were someone who was naturally curious and probably had a high IQ. If you continued programming, it's because you really enjoyed the process and had a knack for it. You understood how the computer really worked, and that didn't scare you, it thrilled you. We didn't have "testers" and "UX Designers" and "QA people" back then, so if you produced a good program at all, it's because you had a high level of conscientiousness. Today, that description fits only a tiny minority of the programmers that I know, just like it fits only a tiny minority of the general population. Those programmers all have the same complaints that you do, and they all have side projects that they work on because the work in their day jobs is unfulfilling. As someone here pointed out, this is a relatively new industry, and I think we're just seeing the same thing that plays out in any industry once "everybody" starts doing it. Just like vacuum cleaners, mattresses, doctors, or oil paintings, there are a handful of great ones, a whole bunch of mediocre ones, and some really awful ones. The bell curve has arrived and is here to stay! My belief is that if you want to have great software, then make it yourself. Don't wait for the majority of others to produce it, because they never will.
  • JoeAltmaier 8y ago
• ern0 8y ago
  We should split layers and rush out everyone from the room where low-level things are going. Then, at least, the low-level things will not break. In an ideal world, we should have building bricks, which even end-users should build an app. Instead, we have frameworks, which are only for programmers, and provide only some automatizable patterns (e.g. a web framework finds which code should be run based on URL), or wrappers (e.g. ORM). Frameworks and libraries _literally_ hide stuff from progrmmers. They even don't know how computers work. I've written some 256-byte intros, and showed them for my colleagues. They were amazed: - Wow, which language do you use, Java? Oh, no you must probably can not use even C++, only C. (Solution: you can not use any language, only assembly.) - Which framework do you use, Unity? Or pure OpenGL? (Solution: you can't use any, you must put every pixel yourself.) They don't know, what fits in 256 byte. They don't care what they're producing. And it's not their fault.
• loilo 8y ago
  > We haven’t seen new OS kernels in what, 25 years? There's actually one in active development. www.redox-os.org
• borplk 8y ago
  Shit quality of software is easily explained with this formula. Because unlike in many other industries there are no strict regulations in software, there is no floor for the quality of the products. So economic forces push quality to the lowest they can get away with. It's much easier and cheaper to apologise to a customer for a bug and give them a refund than to ensure ahead of time that the bug doesn't happen in the first place. The worst case scenario is not all that bad so generally businesses only bother to keep the quality high enough to compete but not "unnecessarily high". If it costs $10k to make an app. You could also always spend $100k to make it more performant and robust. You could also go the other way and only spend $1k and have a slower buggier version. Where do you the line? We end up reaching an equilibrium state. Over the long term things get better incrementally. Individual vendors focus on their core objective and rely on those 'free' improvements. For example a game maker whose game is a bit slow may make a 1 month effort to make their game faster and do what they can. But it's not practical for them to spend 1 year to achieve some sort of perfection state.
  • sha90 8y ago
    This isn't an economic issue, it's an efficiency issue. There are 3 competing ideas: Fast, cheap, quick. You get to pick 2. We're picking cheap and quick because "fast" comes for free on a longer-term timeline. You can argue with whether or not this is the right choice, but this is the choice society is making, and IMO it would work better than taking significantly more time to build something that is only 10% faster. It's more efficient to let hardware manufacturers solve the fast problem when the differences are on the order of 5-10% a year but the time-to-release problem is on the order of 50-100% differences in development times.
• sha90 8y ago
  Article tl;dr: "I believe that apples are more efficient than oranges after all." I just want to point out that the idea that "modern buildings use just enough material to fulfill their function and stay safe under the given conditions" is fundamentally at odds with the author's subsequent thesis. Modern buildings don't use "just enough material", because "just enough material" would be _just_ concrete everywhere; it would have been "just wood" 200 years ago, but that's not good enough now. This is exactly the problem: it's not software that is unnecessarily bloated, it's software that has evolved to solve higher order problems, ones that are not simply based in how fast a computer can count to 10. Similarly, the definition of "fulfill their function" in the context of buildings have changed too. That definition changes all the time, even in building codes. In the modern-building hierarchy of needs, we are way passed the "stay safe" level. We still optimize for safety, sure, but that hardly accounts for your spray foam insulation, HVACs, builtin wireless units, complex built-in cabinetry, complex appliances, and more. Simple things like "electricity" are now part of the definition of fulfilling building function. Go find a 200 year old building and you will find a building that simply does not fulfill today's functions. Even safety standards have and continue to change all the time. You can look at historical building codes and see evolving fire safety (asbestos? NO ASBESTOS!), seismic safety, and more. This is the point. To say that Windows 95 is 30MB discounts the years of improved process space isolation, memory protection, Spectre mitigation, that, if missing, would cause enormous public backlash about why Microsoft doesn't care about security. Windows 10 is 100x larger because WE asked for it to be. WE wanted WiFi, VPN, IPv6 switchover and tunneling all added to our network stack. WE wanted GPU enhanced UI threads. WE want haptic feedback and touchscreens and predictive text and predictive multi-touch pixel accuracy for our touchscreen laptops. This extra complexity exists because our standards changed. A text editor isn't just something that renders ASCII anymore-- heck, it's not even just for rendering characters. My "text editor" is a full web browser because I _need_ that for development these days. Extra complexity is a feature not a bug. We built computers specifically to do this stuff, not in spite of it. The abstractions and complexities aren't getting in our way, they are literally the things we are building. Does performance suffer? Maybe, but that's because we are explicitly paying into functionality. If you wanted a fast text editor, obviously a black-and-white screen that only renders 256 characters will be faster than VS Code, go ahead and use that software, but you're not getting the other things you probably want. Your very next complaint will undoubtedly be "how do easily I diff my Git branches?" -- and this is how software becomes more complex. Welcome to progress.
• fheitzmann 8y ago
  One of my friend pretend that we - software engineers - should work as mechanical engineers. They have a documented, standardized, and limited set of screws, nuts, ... and they build anything with it. This allow to develop CAD tool to assemble, verify, simulate, ... On our side, we have thousands of implementations of basic SW blocks (fifo, mutex, ...), or even "bigger" concepts (server, reader/writer, state-machine ...). What do you think ?
  • majewsky 8y ago
    When we try to standardize software development, we usually end up with people mindlessly chaining together GoF patterns into AbstractSingletonProxyFactoryBeans. I don't know why mechanical engineering is different in this regard from software "engineering". Maybe it's because the domain is more restricted (software concerns basically everything, so it's hard to come up with standardized patterns that are still widely applicable). Maybe it's because mechanical engineering is constrained by the laws of physics, whereas software engineering is mostly constrained by the developer's imagination (and resource constraints, but Moore's Law took care of that one for a long time).
• s-sols 8y ago
  Absolutely agree with the author. I care about my program's efficiency because some day somebody compares my software and competitors and write an article that my software is faster than other. It's a direct influence on my business. Furthermore, I like to write efficiency programs. It's my engineer nature. It's right for me.
• wingtangwong 8y ago
  I agree with the author’s statement of the factual elements, size of binaries, memory utilization, load speeds, resource intensive simple task applications. But I disagree with the author’s reasoning and line of thinking regarding the how and why. The functionality and tome period being referenced for when things were simple and fast referred to a period when your interface was lot resolution or entirely text baed. The action of updating information on the screen and the information on the back end was minimal. Contrast that to today, where you are dealing with a large display with high resolution imaging. You are now talking about a full blown HD gui where your screen updates includes plenty of images, videos, and on the back end, you’re handling edits that may or may not even be local resources, subject to network latencies. Also, modern computers are basically all multi-user systems under the hood, configured by design to be able to recognize, load drivers for, and initialize any and all devices you might deem fit to stick into the computer. Your text editing is running at the same time the system is performing back end scans, probably running multiple web pages, mini services, etc. Not saying there isn’t inefficiency or that there isn’t really bad design. But rather than look at it as meaningless, understand the underlying mechanism for why. Why has no major project taken on the task of a new lean kernel? Why? You can existing kernels and slim them down. Open source means being able to take kernel source, reconfigure it to remove the bulk of the cruft you don’t need, and shrink down your footprint. It also means you OS doesn’t need to be the size of one or more DVD(s), you can slim it down. Same thing for compilation toolchains. You can slim it down. You don’t need to include every library you’ve ever loaded onto your compile environment. There ARE lightweight text editors out there. The heavy weight ones are because people wanted to support all kinds of fonts, formatting, input/output file formats, multi-media, etc. You want to pare that down, you can. Sure, there are lazy coders out there. But what one person calls extra cruft, another person calls tech debt waiting to happen. So you write a super light weight piece of code that doesn’t need to incorporate any special libraries because you wrote it all using standard library calls. Yay. Who is going to maintain that when you leave? You write an application component and forego all of the build toolchain and frameworks that make up so much bulk now, so you hardcore everything. That’s a maintennace nightmare when it gets deployed and other people need to integrate it into a productional stack because no one ever bothered to ready it for CD/CI. Yes, there is alot that sucks about software design at large. But the author is painting with a broad brush, focusing on one set of pet peeves, as observed, and ignoring the underlying reasons for why those design decisions might be in place. For build environments: npm, python, runy, golang, etc: you can configure the build environment for your piece of code to require a specific version of npm, dependencies, etc. this would freeze the runtime environment so that as the language at large advances, you aren’t forced into a broken state. This applies to OS package management as well. The example of DOS being able to run on modern hardware... that is only seeing one facet of the situation. Yes, DOS itself, which is x86 code, can run on modern hardware unchanged.... but that isn’t because of excellent coding. That is because of the massive effort put in by the chip makers to maintain backwards compatibility. In that example, the bulk and legacy stuff just got offloaded to hardware. :/ it works not because it’s great, but because the hardward was designed to allow it to keep working. Much in the same say that some of the more visible software bloat exists: backwards compatibility, cross compatibility, wide array of hardware compatibility, file and data format compatibility, language compatibility, etc. The author makes some good points, but I think that the author also has some tunnel vision going on. Can UI be improved and be less bulky? Hell yes. Can applications return to their simple roots and be highly performant and low latency, hell yes(as long as you are willing to pare down on optional functionality) - can we tun fast OS environments on “older” hardware? Yes! But with the caveat that the price you pay is using older libraries, older methods, or having to back-port code yourself.
• citrin_ru 8y ago
  One reason for software bloat is DRY principle. From very first steps every programmer told DRY! As a result modern software usually has many abstraction layers (because with many layers you can reuse more code) and hundred of dependencies (instead writing/copying a few lines of code we just add one more dependency, which has it's own dependencies and so on).
• d2p 8y ago
  My colleagues (and probably twitter followers) are always sick of me ranting about stuff like this. A few years ago I decided to compile screenshots I'd posted on twitter in the previous single month: https://blog.dantup.com/2016/04/have-software-developers-given-up/ https://blog.dantup.com/2016/04/have-software-developers-giv... That it happens is bad enough, but the fact that so few care about it is reslly depressing :(
  • jakubp 8y ago
    It's an awesome collection you have there, thank you. (And yeah, also a bit depressing...)
• brandonpollack2 8y ago
  Finally, someone I agree with on this topic
• gilday 8y ago
  Software is typically written to serve some business need, and when crappy software serves that need it's incredibly difficult to convince the organization to invest more in quality. However, being asked to crank out half-baked features, poorly tested code, and poorly designed systems takes a toll on programmers who want to be proud of their work. So what is a programmer to do? I feel like buyers and sellers of software are each cool with the status quo, so who has incentive to improve?
• trogan42 8y ago
  I can think of many reasons to deliberately create software that is inefficient. Number one on the list is to make it more reliable. The number one reason software is rarely reliable is because code is more often than not complex to the point where the developer unsure whether it will work or not. In turn, this occurs most often when trying to maintain state via flags. If, instead of trying to maintain state, the software is designed so that the current state of objects is always a function of the original state of the object at the time the App started, you have an App that is potentially slow but like to be extremely reliable and extremely easy to read. I always program this way these days and my Apps are always reliable. For example ToteRaider (see Toteraider.com), a complex gambling App, released 2.5 years ago is still on version 1. Zero defects in production and also zero defects in testing.
• Reverend_Jim 8y ago
  I couldn't agree more. I was a maintenance programmer/developer for 30 years and saw too much of this. It disgusted me to the point where I just rewrote entire systems during slow periods (unauthorized and on the sly). The Amiga OS had full multi-tasking with robust, inter-process messaging, and a full-featured windowed GUI that fit onto one 720K floppy so there is no valid reason that we need multi-gigs for the current version of Windows. I'm posting a link to your excellent article on Daniweb to encourage discussion. Reverend Jim (on Daniweb)
• SeattleCpp 8y ago
  A wise and cynical manager pointed out to me that software purchases are driven by features, not quality. You don't get to assess the quality of software until after the purchase. All other (feature) things being equal, the first product to market gets the majority of the purchases. So dev teams are in a race to the bottom regarding quality. As long as software is purchased, as long as rival companies compete to provide software, this is an iron law. To improve software quality in a race to the bottom, we must improve the quality of (young, inexperienced) software developers. But companies don't want to pay for high quality developers, they are content with barely-qualified devs earning the lowest possible wage. They hire young. They hire offshore. They hire minimally skilled talent. There are places where quality still lives: operating systems, compilers, databases. Places where the software cannot perform its mission if it fails frequently. You have to find one of these jobs, or suppress your gag reflex.
• Yebo 8y ago
  Excellent. You could have mentioned the tremendous amount of energy wasted by servers and data centers to drive the mess. Is there such a thing as Code Ecology?
• n0w4n3 8y ago
  It really looks like what you are looking for is Software Craftsmanship. Several books have been written tackling this topic. You can watch Robert C Martin (aka Uncle Bob) and others (Sandro Mancuso) talking about this problem and how to address it.
• wofkwro 8y ago
  functional programming saves the world. learn that.