14 ms·

Writes large correct programs (2008)

• readthenotes1 2y ago
  I'd prefer maintainable programs of any size. Other than non-trivial academic samples, the odds of a program needing to change over its lifetime or large, and it's current apparent correctness has little to do with someone else adapting it to the ever changing environment. The number of times I've heard "it seems to work and we don't dare change it" is far too many
  • jffhn 2y ago
    >"it seems to work and we don't dare change it" What they mean is: "we don't understand it and we don't have good tests, so there is a high probability that it doesn't work and that doing even the most trivial and seemingly harmless modification would cause an issue to surface, so we don't dare to change it else we wouldn't be able to pretend that it works anymore and might have to fix a lot of issues that we would have a hard time to even understand"
    • azeirah 2y ago
      Yeah.. The dumb thing is that it isn't even _that_ hard to fix this kind of stuff... It does take time and commitment though. But _hard_? No.
      • riehwvfbk 2y ago
        But that is hard. I think you (and many software developers) are using the word "hard" to mean "intellectually challenging", as in "Leetcode Hard". But things that require a lot of effort, time, and coordination of people are also hard, just in a different way. Imagine a codebase with a wart. And yes, without enough tests. Let's say the wart annoys you and you want to fix it. But first you have to convince your employer to let you spend 6 months backfilling missing tests. In the meantime they will pay your salary but you will not work on the features they want. You will be working on fixing that wart. Convincing management: easy or hard? OK, so you got them convinced! Now you can't just fix the wart. First you have to slog through a big refactor and write a bunch of tests. Staying positive while doing this for 6 months: easy or hard? Do you stop other teams from writing more code in the meantime? No, so does the new code come with tests? How do you make sure it doesn't depend on the old "warty" interface? You need a compatibility layer. You need to convince other managers to spend their teams' cycles on this. Easy or hard? OK, the refactoring is done. You release the new software. But, despite all your efforts you overlooked something. There's a bug in production, and when a post mortem is done - fingers point at you. The bug wasn't introduced in pursuit of a new feature. It was part of an effort to solve an obscure problem most people at the company don't even understand. To them, the software worked before, and it doesn't work now, and it's always those nerds tinkering with stuff and breaking things. Convincing these people to let you keep your job: easy or hard? Perf review time. Your colleague shipped a new feature. You shipped... that thing that broke prod and nobody understands. Getting a raise: easy or hard? And that is why these warts fester. The end.
        • polishdude20 2y ago
          The first easy or hard question should be "There is a wart that annoys you, is it really that bad?"
      • jffhn 2y ago
        Some people learned to be fearful. At some point I was working on a piece of software we knew inside out, had good tests for and often ran through hand curated stress tests for benches, analysis or just exploratory testing, so we had a high confidence in it and in our ability to modify it quickly and successfully. Some day executives were visiting and we had to do a demo of our system interacting with another one. Until the last minutes we were happily modifying our code to make the demo better. A guy from other system's team saw that, freaked out, and went straight to our boss, who then laughed with us at how scared the guy was. It turned out his team was not at all that confident in their system.
      • marcosdumay 2y ago
        Will you have time to commit to it? If it's in a professional setting, it's most likely to not be a hard problem, but actually an impossible one.
      • dragonwriter 2y ago
        It’s a hard social problem in many contexts, even if it is not a hard technical problem.
  • necrotic_comp 2y ago
    I get this a bit at my job, and I think there's a difference between making changes (which I do a lot of) and being confident in the changes that you're making. The environment I'm in is completely fault-intolerant, and we're currently hamstrung by our hardware (e.g. no backups/no secondaries/etc.) so changes that we're making have to be well-reasoned and argued before they're put in. Some people take that as being scared, but it's more like "you have to have made this work and tested it before putting it in."
• limit499karma 2y ago
  > how to organize software so that the complexity remains manageable as the size increases So John is missing the role of software architect here. Science, art, and development - 3 roles. Not all visits to the stratosphere are misadventures.
  • kosolam 2y ago
    Yet, some visits to the stratosphere are misadventures.
  • pphysch 2y ago
    I think TFA is implying that good SWEs are good architects too, the skills go hand in hand. I frankly don't believe in the "software architect" as a separate role. I've worked with "architects" who are clearly just BS artists because they know the jargon but have no skill to back it up and make difficult technical decisions regarding tradeoffs.
    • intelVISA 2y ago
      How do you propose those "BS artists" feed their families in an alternate reality tech industry where only the roles that add value are deemed worthy of food and shelter coupons?
      • default-kramer 2y ago
        A good first step would be let them keep their jobs, but strip them of any authority. This already happens at some companies - the architects provide guidance which dev teams are not required to follow. I think it's absolutely insane that we live in a world where many people with an "architect" title don't write code, and sometimes have never written code in their life! That would be like a world full of chess coaches who don't play chess! They just read BS articles like "Skewers are the new forks" or whatever.
        • BobbyTables2 2y ago
          It’s funny, the most effective “architects” only have “developer” in their title. The ones with “architect” in the title move on to the next project as soon as the previous is started…
• tsujamin 2y ago
  This reminds me strongly of reaching the final year industry projects in my software engineering degree, and seeing a significant portion of my colleagues unable to develop software in any meaningful way. There was a curriculum correction in the years afterwards I think, but so many students had zero concept of version control, of how to start working on a piece of software (sans an assignment specification or scaffold), or how to learn and apply libraries or frameworks in general. It was unreal.
  • intelVISA 2y ago
    Is that unique to software though? Plenty of people can follow a plan but still find it tough to start from first principles, I would think.
    • SkiFire13 2y ago
      I've worked for a bit in an engineering company and I was surprised at how bad they were at versioning their documents.
      • codazoda 2y ago
        Do you have any examples of correctly versioned documents?
        • josephg 2y ago
          I mean, I’d say a markdown / latex / typst document in a Git repository would fit the bill. I’m working on a history project at the moment which has reconstructed the version history of the US constitution based on the secretarial records and various commentaries written during the drafting process. At the moment we’re working on some US state constitutions, the Indian constitution, Irish peace process and the Australian constitutional process. We only have so many historical records of the committee processes, but it turns out to be more than enough to reconstruct the version history of the text.
          • 2y ago
• deleted 2y ago
  [deleted]
• brap 2y ago
  One thing that’s missing: programs are mutable. A good programmer writes programs that are easy to maintain and extend.
  • billy-ilograph 2y ago
    Agreed, a sign of good programming is that the program feels easy and natural to extend. But there is a corollary, I think. A sign of good software development is that the program hasn't been extended in "unnatural" ways. That speaks to the developer's discipline and vision to create something that was fundamentally relevant to begin with.
  • alserio 2y ago
    Also, programs that are easy to extend will be extended until they are not. I don't remember the name of this Law.
  • andrewflnr 2y ago
    Arguably, that's implicit in "large". A large program has most likely been extended several times, through its various stages of growth (the alternative being that it was written in a single pass, unlikely). This is tied to the author's point about managing complexity: complexity is the main enemy of maintainability.
• herodotus 2y ago
  When I was a prof (many years ago), I was working on a database for a political campaign. The code was a mess. I asked a couple of my colleagues (successful comp. sci. profs) to help out. It became clear very quickly that there are two kinds of comp. sci. profs: those who can program and those who cannot.
  • Thorrez 2y ago
    One of my computer science professors when his laptop wasn't connecting to the projector: "I hate computers."
    • saulpw 2y ago
      That's one of the ones who could program.
      • downut 2y ago
        Oh dear this hits way to close... Get deep enough into the machine and you come to expect the (in this case minor) disasters.
    • cornel_io 2y ago
      That sounds like the correct response if he knows anything about the field: shitty peripherals are an unsolvable problem from top to bottom. The hardware sucks, the software sucks more, there's absolutely nothing you can do to fix it as an end user (unplug it and try again is where "debugging" ends), and the production chain will never improve along the quality axis because the margins are tiny and people are very price sensitive.
    • coliveira 2y ago
      Why do you think people become CS professors? One of the reasons is that they don't like working in the industry and maybe don't even know how to program.
      • cma 2y ago
• keybored 2y ago
  I can definitely write a 1KLOC program to solve a 10-line problem.
  • Thorrez 2y ago
    >(When I talk about a program that is so many lines long, I mean a program that needs to be about that long. It’s no achievement to write 1,000 lines of code for a problem that would be reasonable to solve in 10.)
    • keybored 2y ago
      That is indeed the self-defeating/deprecation part.
• harimau777 2y ago
  It seems to me that in other areas of tech, companies generally hire electrical engineers, mechanical engineers, civil engineers, etc. On the other hand, software companies feel that they don't need to hire computer scientists. Then periodically there is a discussion on Hacker News that boils down to "all of the other engineering disciplines can make reliable predictions and deadlines; why can't software?" or "why is this company's code so shoddy?" or "why are we drowning in technical debt?". Perhaps the these are all related?
  • bckr 2y ago
    Is your working definition of a computer scientist similar to a civil or electrical engineer? To me, a computer scientist is someone who studies computation. They probably have the skills to figure out the run times of algorithms, and probably develop algorithms for solving arbitrary problems. A software engineer is what I would call someone who can estimate and deliver a large software application fit for purpose.
    • pphysch 2y ago
      I agree with this. A reason there is so much crappy software is because companies are hiring fresh CS grads expecting them to do real software engineering work. And they end up hacking it like they hacked it through school. CS programs have gotten better at teaching real SWE skills, but the median CS grad still has ~zero real SWE experience.
    • harimau777 2y ago
      Theoretically, I think that the ideal situation would be for a Computer Scientist to be someone who performs fundamental research in computation while a Software Engineer (I think that something like Computer Engineer would be a better term) applies computation research. Analogous to the relationship between a Physicist and an Electrical Engineer. However, as the terms are currently used, I see Computer Scientist as analogous to Electrical Engineer. On the other hand, it seems to me that Software Engineer is used to suggest that developers don't need to know the theory behind computation. Therefore, I currently think that the way "Software Engineer" is used respresents a lot of what's wrong with current software development.
• PaulHoule 2y ago
  There are many axes of complexity. Routine line of business systems, say an inventory management system for a car dealer, with a proper architecture costs should be additive instead of multiplicative, a certain cost to develop features such as “autocomplete control that draws values from a database row” and a certain cost to deploy that control. Double the number of screens using the same features and you double the cost but it feels like sub linear scaling because for the second tranche of forms you don’t have to redevelop the components and it can go much faster. That ideal can be attained and you can be in control in application development but often we are not. When you are in control the conventional ideas anout project management apply. As you get very big you start having new categories of problems, for instance a growing social system will have problem behaviors and you’d wish it was out of scope to control it but no, it is not out of scope. Then there are projects which have a research component whether it is market research (iterate on ideas quickly) or research to develop a machine learning system or develop the framework for that big application above or radically improved tools. A compiler book makes some of those problems look more regular like application programs but he project management model for research problems involves a run-break-fix trial of trying one thing and another thing which you will be doing even if you are planning to do something else. Livingston, in Have fun at work says go along with the practices in the ocean you swim in (play planning poker if you must) but understand you will RBF are two knobs on run-break-fix: (a) how fast you can cycle and (b) the probability distribution of how many cycles it will take. Be gentle in schooling your manager and someday you might run your own team that speaks the language of RBF. Unit tests put a ratchet in RBF and will be your compass in the darkest days. They enable the transition to routine operation (RBF in operations is the devil’s antipattern!) They are not a religion. You do not write them because a book told you so, you write them for the same reason a mountain climber wears a rope and if you don’t feel that your tests are muda, waste as they say in Japan.
• PaulHoule 2y ago
  Computer scientists advance in their careers by writing papers, software developers do by writing programs. Some CS grad students and profs are genius programmers, but mostly CS researchers write a program that at best lacks the polish of a real product and at worst almost works. When I was in physics grad school I had a job writing Java applets for education and did a successful demo of two applications at an CS conference in Syracuse and was congratulated for by bravery. I was not so brave, I expected these programs to work every time for people who came to our web site. (Geoff Fox, organizer of the conference, did an unsuccessful demo where two Eastern European twins tried to make an SGI supercomputer show some graphics and said “never buy a gigabyte of cheap RAM!”)
  • aleph_minus_one 2y ago
    > Computer scientists advance in their careers by writing papers, software developers do by writing programs. Rather: Computer scientists advance in their careers by writing papers, software developers do advance in their careers by becoming managers. :-(
• Missanna 2y ago
  [flagged]
• devjab 2y ago
  I’ve worked on and finished some extremely large programs over my years in non-tech enterprise. I’m also an external examiner for CS students, and I’ve regularly talked about how I think the curriculum is outdated. In Denmark where I’m from it’s rare to get a programming application from someone without a related degree. It wasn’t so rare 20 years ago, but I can’t remember when I saw one last. I agree that degrees, and especially CS, doesn’t guarantee that people can code. We use them mostly as a “safety net” in our hiring processes here in my region of the world. Basically you can view hiring the wrong person as the most expensive mistake you can make as a manager, and educations are a sort of risk management. You might think that the “can this person actually program” risk is worst for people who are fresh out of their education but there are a lot of factors which can play into it. Older devs may be set in their ways, maybe even religious about some sort of programming philosophy. On the flip side they will start producing value right away. Anyway, by far I think the biggest hurdle in our industry right now is pseudo-jobbers like project managers, business process owners, scrum masters, various architects and what not. Not everyone is a waste of time, some of them do excellent work and function as exponential productivity catalysts. The vast majority of them, however, spend so much time engineering the process, architecture, whatever that their teams never ship on time or within budget. In this sense I think “correct programs” is hard to value. Because often the “incorrect large program” that doesn’t scale, will be much more valuable for a business than a “correct program” which never even gets the chance because it took to long to get out there.
  • intelVISA 2y ago
    Those pseudo-jobs are a necessary evil in lieu of some sort of UBI imo. Plus they're useful for sabotaging your competitors' TTM.
    • RealityVoid 2y ago
      I believe we're not at the productivity level that UBI requires, and those "pseudo-jobs" are a waste that could be redirected to some more constructive endeavor. Even if you oppose consumerism and building stuff or whatnot, that wasted effort could be directed towards making products more sustainable or making better recycling supply chains or building nuclear power plants or whatnot.
• kleiba 2y ago
  I graduated in CS at a renowned university over two decades ago. The faculty was very theory-oriented, basically every lecture was just theorem, proof, example - even the programming lectures. As a matter of fact, there was only a single lecture I took (and which I didn't need to take) where we needed to use computers for the weekly exercises.
• hnthrowaway0328 2y ago
  Can absolutely relate and understand. I taught myself C++ by writing games with SDL2. The first game -- snake took about a couple of hundred lines and I put everything in one .CPP file. And I felt pretty good. The second game, well, I forgot what it is, not Tetris nor Breakout, but it was complex enough that I realized that I need to put code into header files and source files. The last game of that project was a Ultima-spinoff. I even used the same sprite sheet. The complexity completely drowned me. I was constantly asking myself how should I arrange things properly so I don't need a lot of global variables that every file can see, because naturally the Game class needs to see and know all other classes, and the Renderer class needs to see and know many other classes too, etc. Eventually I dropped the project. A few years ago I picked it up again and figured out something that is close to Entity - System (not ECS just ES). I didn't complete the project but then firmly believe that it was the right architecture, and I was simply too burnt out to complete it. This year I learned about ECS from pikuma. I think it's over complicated for small-medium games. Anyway I'm trying to say that I agree that writing a 10,000 line project is way more complicated than 10 1,000 line projects.
  • wavemode 2y ago
    Unrelated to the central point of your comment, but I've also found that a simple entity system is usually perferable to ECS for smaller games. ECS aids performance, but performance usually isn't what you're struggling with in a small indie game. You're mainly just struggling to organize your code, and just need something simple to help manage complexity.
    • LoganDark 2y ago
      Minecraft is an entity system and it seems to work just fine. (Fun fact: Forge Mod Loader makes Minecraft an entity component system.)
      • immibis 2y ago
        having things called entities and components isn't doing ECS. Minecraft is a traditional OOP class hierarchy of entities, which (with Forge) have a Map<String, Object> whose entities are called components. In the technically superior Fabric ecosystem, you can add fields directly to the entity class using bytecode modification instead of having HashMap overhead on everything. Accessing them is a bit roundabout, but in a way the JIT compiler can optimize.
• MrMcCall 2y ago
  The key observation about the Dunning-Kruger study is that humility, hard graft, and honest perseverance towards improving oneself are the three most important factors in achieving proficience in crafting large software data flow systems. How one designs and implements such systems is the art of any form of creative endeavor. Designing and implementing large and correct systems is a matter of growing them, from small, trusted pieces into larger interconnected systems of systems, with ever greater care, knowing that the entire thing can collapse at any time if the wrong decisions are made or have been made.
• PaulRobinson 2y ago
  When choosing my degree back in the mid-1990s, I chose a BEng in Software Engineering, and not a BSc in Computer Science because I wanted to enter a career writing software, not a deep study of the theory of a branch of applied mathematics. I was fortunate enough to have figured this out for myself, and whenever I met a CS grad in my early career it was obvious that the production of actual software terrified them. Meanwhile I'd learned how to build (and how not to build), working programs in C including a simple OS on an M68k chip on a VME bus. I struggled with my final year project because it became too theoretical and CS-ish (trying to write a Prolog to SQL interpreter), so my grade took a hit, but I am really glad I entered industry with useful, practical skills that employers valued. There's always going to be a place for pure CS, I'm glad it exists as a discipline, but more kids should understand the difference, and more colleges should be offering degrees that teach people how to software is built (and how to build software yourself), not just write papers.
  • ahartmetz 2y ago
    There is a problem insofar that software engineering somewhat exists, but it's more like CS for dummies. I think it's important that it actually teaches at a high intellectual level, just about something different. Let the students write an operating system, not some typical "manageable student task" bullshit. I've had an exchange semester in Oslo where they asked their students to do just that. First exercise: bootloader in assembly, second exercise: task switcher, third exercise: virtual memory, fourth exercise: filesystem - etc. It might have been one or two weeks per exercise, not sure. I actually failed most of the later exercises due to "other priorities", but I strongly felt that they were doing it right. The exercises were of course simplified, but a bootloader is a bootloader, task switching is task switching, virtual memory is virtual memory... no flourishes and some fixed buffer sizes etc, but the basics were there.
• andrewaylett 2y ago
  Computer Science is to Software Engineering as Physics is to Civil Engineering. Software Engineering is to programming as Civil Engineering is to construction.
  • cornel_io 2y ago
    I don't think that's really true in practice, though at a competence level it absolutely is. Except at the biggest places, even low level programmers are both empowered and expected to make architectural decisions that have big impact on maintainability, testability, and performance. Addition of a bad-but-fast coder to a team is the worst possible outcome because they are really productive at screwing up the whole codebase in ways that will take even more time to peel apart. It's actually worse if their code is in the "functional but ugly" category, since PMs love those people as long as they're really fast, and they hate when other engineers complain about their work. In construction, the random guy pulled in to swing a hammer may cut something wrong once or twice and waste a little time and money, but he's never going to be trusted to design the way the support beams hold up the whole house, so the damage is very limited. Software is not like that, we expect everyone to do some amount of design and engineering, whether or not they have any ability to do so. If as much software dev was driven by very low-level grunt work as is the case in construction, LLMs would already have revolutionized the field a lot more than they have; as it is, we've probably got another couple years to wait.
    • andrewaylett 2y ago
      This is true. Construction has had millennia to work out how to best organise its different disciplines, though -- I hope that it won't take software that long, nor that it'll need to be quite as stratified. Software Engineers are usually quite capable programmers, while construction needs a lot more people _doing_ the construction than working as Civil Engineers, even if Civil Engineers wanted to join in. That's not quite my point, though: we don't expect physicists to be good at Civil Engineering or construction, and we ought not expect Computer Scientists to be good at Software Engineering or programming. Having some understanding of physics makes a Civil Engineer better at their job, similarly for Computer Science and Software Engineering. And both construction and programming can be undertaken in isolation, but you're unlikely to be successful in your larger project unless you have Civil Engineering or Software Engineering expertise.
• jongjong 2y ago
  My non-technical co-founder decided to teach himself coding in the past couple of years. He can write quite complex logic but he tries to make everything as generic and flexible as it can possibly be and so his code is very hard to read as it invents a lot of concepts. He had to refactor it many times due to bugs or unforeseen technical limitations. Took him months to write. On the other hand, I wrote a script which does a similar thing as his but for a different environment (over a network with added latency, bandwidth limits, network instability) and only covers the essential cases but it only took me about a day to write and has been working without significant flaws since the beginning. Only had like 1 bug for a minor edge case. Also, my code is very short relative to his. This experience reinforces a rule for coding that I've had since the past 5 years or so. It's basically: "If you can't explain what your code does, from start to finish, in simple language to a non-technical person (who has the required business domain knowledge), and in a way which answers all questions they might have, then your code is not optimal." Abstractions should always move you closer to the business domain, not towards some technical domain which you invented and only exists in your head. The part about "start to finish" is important but doesn't mean "every line". You should have abstractions which may be beyond the non-technical person's understanding but you should be able to explain what each abstraction does at a high level without jumping into each function. You should be able to walk through any major feature by walking through your code, without having to jump around many files and without having to pause constantly to explain abstractions as
• jongjong 2y ago
  I think also, being able to write a minimal amount of code to implement any given feature is important. However, your code should anticipate a range of possible future requirements changes... But then it shouldn't try to be a silver bullet either. Experience helps a lot as it allows you to see hurdles and limitations ahead of time so you know exactly how much silver you can put in your bullet for any given part of your code.
• brunoarueira 2y ago
  When I was in the college, I'd made some consultancies to final students from CS degree which don't know how to program and even bad, the project was a simple copy and paste some classes, change attributes and manually test it. So, the main problem I'd seen was they don't have the ability to figures out the system and it's a basic expectation!
• dang 2y ago
  Related: Writes large correct programs - https://news.ycombinator.com/item?id=2556270 https://news.ycombinator.com/item?id=2556270 - May 2011 (32 comments)
• lincpa 2y ago
  [dead]