27 ms·

This is a new summary of the real-life case that inspired me to go do PhD studies in information systems. I wanted to understand why IT incompetence still could

• IronWolve 3y ago
  I did a quick look around for some blogs about this early paper to digital transaction register migration, I didn't see much for such a major case. Just a few basic things that wasn't included, no audit/transaction logs, transactions modified by tech support to keep the system running. Operators couldn't prove they didn't steal funds, and the british law that computers systems are to be trusted as fact, pretty much convicted them all.
  • imtringued 3y ago
    This isn't a software error and it is pretty clear. There are two problems here. First, the branch manager is responsible for calculated shortfalls, even if the software is broken. Second, there is no way to overturn broken software. Third, the prosecutors are overzealous in trying to shut these people up and convict them straight away. The software itself was just a convenient medium for abuse of authority.
• porker 3y ago
  > I quickly found out that IS research had already figured most of this out What is "IS" in this context? I did some Operations Research modules at uni and thoroughly enjoyed it, but it had nothing to say about why projects didn't work.
  • onei 3y ago
    Information systems at a guess
• onei 3y ago
  Were there any particular themes that stood out as you when understanding the causes of IT incompetence? I'm hoping there's a less depressing answer than "some people will never care".
  • PakG1 3y ago
    I'd say that the biggest thing that stood out to me was what people call communities of practice. Each professional community has their own knowledge and best practices, but these don't generalize well to other professions. So you have boundary spanners who can bridge two professional communities by being good at both, but those types of experts are rare. Also, the amount of experience and learning required to become an expert in two professional communities, rather than just one, requires such a large amount of time that most people can't be bothered to put in the effort. It's enough work to do one's own job well already. The go getters can of course do it as a natural course of action, but they are outliers. There are a limited number of job opportunities that require developing this experience on the job, so there are limited opportunities to become a good boundary spanner in the first place. Furthermore, people aren't naturally interested in multiple disparate subjects. True renaissance folks like Leonardo da Vinci who are interested in becoming experts in both art and engineering are rare. Elon Musk types that will try to dive deep into multiple unrelated areas are rare. All of this adds up to boundary spanners being rare. As such, leaders who can develop cultures that handle multiple areas simultaneously (see the founders of Flexport who understand both tech and shipping logistics) are rare. In short, expertise is hard to develop, expertise in multiple areas is rare, and coordination between two areas that understand different worlds is difficult without boundary spanners. As a result, you get failures. See any software engineer who creates a startup to try to revolutionize some old-school industry and then fails dramatically because they don't understand the problems that actually need to be solved. The outliers will figure it out, but not everyone can become an outlier due to reasons discussed, among other reasons.
• kortilla 3y ago
  This is not well researched at all. Anything that depends on “if everyone acted intelligently and good will” is broken. If you’re interested in academia I strongly recommend you go back and look at designing systems that function in the face of incompetence and even adversaries.
  • PakG1 3y ago
    You can have your opinion, but I have mine after reading lots of research papers. Obviously, there are ways that things can still be improved. But even the design of systems that function in the face of incompetence and even adversaries will face intractable problems of incompetent leadership and governance. It's one thing to say "this is how it should be done" and it's another to say that we've managed to get people to do it the way it should be done. No matter how much improvement we see on ideas for how it should be done, we're still no closer to solving the second part, despite much effort. In the end, there will always be incompetent leaders in charge at some point for some project somewhere, and more often than we'd prefer. You can lead a horse to water, you can't make it drink.
    • kortilla 3y ago
      It’s not an opinion and you can cop out to lame “what if the government is hitler” arguments, but resilient systems are definitely an engineering/science/math problem. The entire field of cryptography wouldn’t even exist if the boundary of research ended at “good actors”.
      • Joeboy 3y ago
        Surely the field of cryptography relies on conscientious and competent actors developing solutions that are robust in the face of malicious actors. I am skeptical that there are software development practices that will allow me to hire a team of feckless incompetents and have them develop quality software. If you know of any I'm interested to hear about them.
        • kortilla 3y ago
• chii 3y ago
  > I quickly found out that IS research had already figured most of this out > People suck, organizations suck, societies suck and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley. The idea is that there should be liability assigned to important systems, for which this liability makes the onus on the creator/owner to build in safe guards, checks or other protections to prevent disasters. Why this isn't applied to software engineering is a whole nother story, but i think it probably should. Move fast and break things is not something i wanted to hear tbh.
  • saurik 3y ago
    Even Facebook realized it was dumb and Mark Zuckerberg apologized to us all on stage at f8 a decade or so ago and said they were now going to say "move fast with stable infra", yet I still see tons of people fetishize and emulate the original.
    • disgruntledphd2 3y ago
      Honestly though, FB wouldn't have won without the move fast and break things approach. But different techniques work at different scales of company. You're doing super well if you're successful enough that some of your values need to be re-written.
      • jacobr1 3y ago
        And even then they didn't pivot to "don't break anything." They still are willing to break user workflows are quickly test new features that have a high risk of being rolled back. They key is that they are able to set more explicit tradeoffs on the risk they want to take relative the benefits and what scenarios they want to optimize. The same applies to all engineering disciplines - arguably managing requirements and tradeoffs to produce a working system is what makes something a work of engineering vs another discipline.
  • stevefan1999 3y ago
    > and yet you don't see very many buildings collapse, bridges fail, and damns flood down valley Computers, at its best existed for only 200 years. That's counting from the difference engine that Babbage never finished building, and if we are talking about the first modern computer that does contribute significantly in a way, that's around the time ENVAC and Z3 appeared, just around 80 years. Structural Engineering existed for over 2000 years. With that comes with a lot of people died in natural disasters and advancing material science to improve structural integrity. But even the Japanese to this day still can't solve the earthquake and the problems the fallout makes. > Why this isn't applied to software engineering is a whole nother story I actually think it is. That's why real-time operating system and mission-critical hardware exists for human to fly in the air and explore the space, not counting a lot of time-sensitive industry robotics software that controls the actuators in real time as well. That said, the strict requirements of real-time programming requires a lot of expertise such as CPU cycle counting, CPU slack reduction, timing requirements and choice of algorithm (such as EDF scheduling and real time memory allocators like TLSF, and you won't see jemalloc on embedded devices, right?), stack or heap memory allocation which also complicates the programming stuff, because removing malloc might free you from OOM but this means a lot of functions needs to accept extra parameters to state the output location, and that means a lot of normal software can't be used. (You can go for a hybrid approach by using arena allocation, but that still isn't a perfect solution) As you see, even for soft real time engineering (which I actually mentioned so far, I don't know much about the real hardcore "hard real time engineering" though), the sheer complexity here already, means there's a lot of design decisions which simply just makes people stay away and just go for normal software engineering (but in the end, if everything is predicable its fine). > Move fast and break things is not something i wanted to hear tbh. When Zuckerberg said that, he's referring to his startup mindset when Facebook was really just a startup in his day. In a highly competitive market environment, startups has to fight desperately for their own survival, even with lots of fundings and VC rounds. Startup can move very fast and their agility is the only weapon against the old dogs. Now you've become one of the old dogs, you don't break things.
• DeathArrow 3y ago
  The biggest issue isn't the software glitch system. It's the legal system that threatened innocent people with prison for theft unless they admit they are guilty for crimes they didn't do and pay for damages they didn't do. This case was brought to public attention and repairs were attempted only because it's huge and involved hundreds or thousands of people. How many disparate cases there are, where people's lives are destroyed and innocents are rotting in jails, we have to ask?
  • PakG1 3y ago
    There are not as many that involve jail, but there are a variety that involve ruined lives and even bankruptcy. A recent example is the Phoenix pay system that was used to pay Canadian federal government employees and contractors. https://en.wikipedia.org/wiki/Phoenix_pay_system https://en.wikipedia.org/wiki/Phoenix_pay_system And I agree that the problem wasn't the glitches. I personally think it was the corporate governance that failed, not the software development and debugging process. The legal system was complicit and enlarged the overall consequences, but the but for test tells me that it was the poor corporate governance that was at fault for a root cause.
  • usr1106 3y ago
    Every skillful programmer knows that all software is crap. Judges, salesman, and managers don't understand that.
    • jack_riminton 3y ago
      A big problem is they don't want to do the work to understand that, which is the exact outlook the PO had... "We need some software, ok let's get a big reputable company in to do it for us, we shouldn't get bogged down with all those horrible technical details"
      • zimpenfish 3y ago
        Not wanting to defend the PO but it wasn't really their decision - it was a PFI (private finance initiative) foisted upon them by the Tory government of the day as one of their recurring "STOP BENEFIT FRAUD!" lunacies.
• madaxe_again 3y ago
  > people of integrity Our current system of the world quite strongly disincentivises honesty and integrity - rather, being a bombastic charlatan with a flexible relationship with the truth will get you anywhere.
• heads 3y ago
  It would be really interesting to study The Post Office in particular. Something about this organisation attracts some very sour people. Or perhaps they weren’t always like this but have become so in my adult lifetime over the last few decades? In the early 2000s there was a TV ad campaign for “The People’s Post Office” where the sub-postmaster role was played by John Henshaw, a character actor known for playing hard bastards and, in his most recent role on The Cops, an exploitative bent copper from Bradford. A strange but apt piece of casting.
  • ponector 3y ago
    Low salary coupled with customer facing job creates sour people. Imagine you are talking everyday to weird, bitter, arrogant, rude customers for years - even with high salary you will be not so positive.
    • ithkuil 3y ago
      Sounds relatable. I also think there may be some selection effect going on: if you didn't have a better alternative than a low paying customer facing job you may be already have a little sourness to start with
• risyachka 3y ago
  Most of the time issues like these are from companies that pay like $50k salary for senior positions. So everything checks out.
• lijok 3y ago
  > why IT incompetence still could exist to this degree in this day and age after all the knowledge the world had developed about good IT and software practices over decades of experience Because we’re not professionals. We don’t profess anything and do not have standards. There is no regulation for our industry and no IT association that can strike you off from practicing this craft. There is no accountability, and when there is no accountability, people naturally regress to either lazy or exciting behaviours.
• andy_ppp 3y ago
  I could give you a huge list but mostly it is computer programmers having the specification constantly changed by management and stakeholders. Even bad software developers can eventually make the software functional, even good software developers can write bad software if the organisation is going out of its way to break everything they do.
  • PakG1 3y ago
    There is definitely a sensemaking process where organizations have to figure out what it is that they really need. But I wouldn't fault organizations for that. Most startups go through the same process trying to figure out product-market fit and you don't see those startups blaming their customers for not knowing what they want.
    • andy_ppp 3y ago
      Most startups die doing this, most large orgs just set first to loads of money and make their software buggy, they already have product market fit.
      • PakG1 3y ago
        If they had product market fit, they wouldn't have major feature change requests that turn the product upside down and inside out. But either way, startups don't blame their customers for being unable to meet their customers' needs. I think it's poor practice to blame organizations for being unable to meet organizational needs, especially when we already know that organizations and users don't know how to conceptualize software requirements well, let alone create software.
        • andy_ppp 3y ago
          I constantly see large companies trying to reinvent the wheel in really haphazard ways, what has been your experience? Mine is I have been contracting in roughly 20 large companies since 2010 and before that I worked at Yahoo! and others.
• o-o- 3y ago
  Some people suck at management, some people suck at coding, and some people suck at self-awareness. I can think of two things that I believe would make a difference in any LargeCorp: First, a standarized way to visualise and execute business logic that allows developers and management to reason together. (The no-code movement is on the right track in fostering a common way to interface with code). And second, a responsible editor for each piece of code. I think a key factor is that software historically hasn't enjoyed industrialisation to the degree of hardware (or construction for that matter). I can buy a standardized CPU of millions of transistors and integrate it into a standardized motherboard with just a snap. We have managed to standardize software up to the OS level, but after that it's up to the developer and her shortcomings. https://www.codevalley.com/ https://www.codevalley.com/ does some interesting work.
• fzeindl 3y ago
  Don‘t think like that. Medicine is a complicated field as well, but after evidence based medicine was invented death rates were going down. It is certainly possible to find and enforce lists of best practices.
  • epgui 3y ago
    The flip side of that is that despite how awesome modern medicine is, we still can't get people to get vaccinated (for free!) and put a mask on their face during an infectious disease emergency. You will find that people just suck no matter what field you work in.
    • fzeindl 3y ago
      Reminds me of a saying from a dear friend: „Humanity is a boat with a big hole and we constantly need to shovel out water to keep it from sinking.“
• ksec 3y ago
  People who are selfless, humble, intelligent, competent AND be people of integrity, are never the people who win the contract for any information system though. >Of course, I quickly found out that IS research had already figured most of this out, and that perhaps, people were just people and crappy organizations were just crappy organizations, and perhaps that's something that will never change because bell curve distributions exist for almost everything. Hence why we need to keep things simple. The human part will never change, or at least change at rate that will take many generations to improve if you are an optimist. I actually prefer things to be Hybrid rather than all-in digital.
• deleted 3y ago
  [deleted]