7 ms·
A $300M IT flop
- jsvine 12y ago> Throughout the past six years, the project has been stuck in the beta phase. According to McKinsey, "for past 5 years, Release 1.0 is consistently projected to be 24-32 months away."
- deleted 12y ago[deleted]
- beat 12y agoI don't think you can extrapolate from a food court on a military base to a $300M project to rebuild Social Security IT. Don't let your political biases blind you to reality.
- deleted 12y ago[deleted]
- bitJericho 12y agoHow dare the government ask (and pay for) companies to engage in ethical and socially concious ways! They should be paying bottom dollar and making sure that the service workers make minimum wage just like they do in the public sector!
- Fomite 12y agoIn 2012, the largest government contractors were: 1. Lockheed Martin 2. Northrop Grumman 3. Boeing 4. SAIC 5. Raytheon 6. General Dynamics 7. Hewlett-PAckard 8. Booz Allen Hamilton 9. Computer Sciences Corp. 10. DynCorp International. Also in there are companies like Dell, IBM, Accenture, Batelle, Deloitte, Honeywell, AT&T, Rockwell, GE... Hardly companies run by blind immigrant union women. And lest you dismiss those top ones as "just" defense, note that I've seen presentations by both General Dynamics and Northrop Grumman talking about their work in the health sector. There's something especially strange about booths featuring pictures of fighter planes at epidemiology conferences.
- deleted 12y ago[deleted]
- adwf 12y agoIt's still got nothing on the NHS £10bn IT flop... I always wonder when I hear about these projects, where exactly has the money gone? It can't possibly be in hardware, that would make some of the biggest data centres in the world just for the NHS. I can't even see how it can be manpower, how many IT people can you get for £10bn? And surely if it were corruption, someone would notice so much money being squirreled away... There's just no logical explanation behind these things.
- arethuza 12y agoI can recommend the book "Plundering the Public Sector" by David Craig that explains how projects like the NHS IT fiasco happened: http://www.amazon.co.uk/Plundering-Public-Sector-Letting-Consultants/dp/1845293746/ref=sr_1_5?s=books&ie=UTF8&qid=1406646332&sr=1-5 http://www.amazon.co.uk/Plundering-Public-Sector-Letting-Con... [NB Note recommended for UK taxpayers who have high blood pressure and/or easily upset]
- martiuk 12y agogov.uk which had a great success, was done by hiring their own people to work on it. I don't see why anyone would think outsourcing is good for public projects.
- beat 12y agoI worked on a high profile government project that had five different vendors involved. FIVE. The inter-organizational management problems were astonishing. I'm pretty sure that by the time I left, managers actually outnumbered technical staff. The biggest problem, though, was the lack of pushback. The government officials would ask for the moon, and the vendors would promise to deliver it. There was almost no one saying NO - the word that project most desperately needed to hear. Features outweighed core infrastructure reliability, so configuration management was a nightmare.
- TheOtherHobbes 12y agoIt's very good for the companies that benefit from the outsourcing.
- jeremymcanally 12y agoYou'd have leaner, smarter firms that could actually deliver something getting involved if the bidding process wasn't so obtuse and opaque. When I was consulting, I responded to a few RFP's but they were always either (a) obviously tailored to the capabilities of a single firm so there was no way to win unless you were them or (b) so ridiculously vague or out of this world impossible ("We need you to basically rewrite GitHub for us and the budget is $30k") that it wasn't even worth the time. Even now where I work, we have to deal with government agencies literally telling us they can't tell us any requirements at all because that would give us an advantage over other firms somehow. We just have to pitch our product to them and hope that it fits, even though at this point in our development cycle we could (and really, want to) make adjustments, feature additions, etc. to fit them if we were allowed to. Government contracting in IT, from my point of view, is nearly 99% about "playing the game" and about 1% actually delivering something. Especially since it seems that even if you fail spectacularly, you still get paid (possibly paid even more to fix your stupidity).
- dm2 12y agoThat's exactly right. The staff organizing the project for the government gets advice (bad advice) on the technologies that it should use from approved development firms then make using that technology a strict requirement. These firms are some of the few that have developers that use that technology and win the bid, everybody except the hired firm loses. Any small competent development team can built almost anything imaginable with 6 years of time, proper funding, and without unnecessary restrictions. $300 million to develop anything is insane. I can't imagine what happens when the check is cut, "let's hire 100 programmers, 10 managers, 5 accountants, 10 support staff and they'll figure it out." Maybe there needs to be some kind of "insurance" company that oversees the project and has to pay back a portion of the money if the project fails. I'm not sure if that's reasonable, just an idea. If the project starts going off-track then they start letting people go and find competent replacements. Plan projects out, have milestones, goals, backup plans. If they spend 6 years and $300 million dollars on a computer system and fail then they were just winging it from the beginning. The managers responsible should be identified and go on a blacklist. Right now it's either "oh well, hopefully we'll do better next time".
- incision 12y ago>'The project was conceived to replace 54 separate components used in the SSA's disability determination system.' This is the bit that jumps out at me as clear indicator of the sort dysfunction that dooms projects like this more so than anything else. It's indicative of a the classic exploding scope and fragmented responsibilities leading to an overall lack of accountability that unfortunately characterizes IT in and around Government. Gershman seems to agree. "If there's not strong leadership, problems arise with coordinating work between contractors, and there's no one person looking at the system as a single entity -- everyone is making sure their piece is done, and fulfilling their responsibility up to a certain point," However, I'm not wholly keen on the the start-ups as a solution direction taken later. It's not that it's wrong, just beside the point unless the start-up savior is specifically targetting culture and process change as opposed to technical implementations. The 54-armed kraken of a defined by committee project will drown and devour a fleet of confused start-ups lost in the mist of *BE requirements and contract mods just as readily as the Lockheed leviathan.
- greggersh 12y agoWhat I meant by startups is that in that world, there's market forces driving you to do things better, faster and cheaper. Not necc. that we need startups to solve the IT problems of government (although that could help), but that we need to create an environment where there's market forces pushing government contractors to do things for less people/less money and better results (which are all very, very possible).
- rayiner 12y agoInteresting quote from the article: > 'You have companies like Lockheed Martin -- they're not web development companies, they don't do work in the private sector, they're not building start-ups,' Gershman said. 'There's very little in the market that forces them to adopt new technologies.' All of these are true, but I think this glosses over some fundamental structural problems with government contracting. The SSA has almost 60 million beneficiaries, and deals with distributing an entitlement that is extremely sensitive and protected by layers of legal process. 8,000 Social Security checks get lost,[1] and its not a blog post, its national news. Moreover, a lot of lean and agile processes just aren't consistent with political reality. There's no minimal viable product, there's no invite-only beta testing, there's no iteration, there's no A/B testing, etc. Technical decisions ("let's just roll out to northern Alabama first") become divisive political ones. Purchasing decisions ("we've had good luck with this vendor before") become the basis for accusations of favoritism and corruption. Perhaps most importantly, there's no failing fast. The private sector accepts the fact that many engineering projects will fail.[2] Good companies take care not to punish failure unreasonably. But in the public sector, failure is just license for those opposed to project to crow: "we shouldn't have been doing this anyway!" If you fail fast, it's easy to pin the blame on you. This creates a perverse incentive to drag a project along so failure happens over years if not decades. [1] https://medium.com/@jan.curn/how-bug-in-dropbox-permanently-deleted-my-8000-photos-cb7dcf13647b https://medium.com/@jan.curn/how-bug-in-dropbox-permanently-... [2] Most big IT projects fail: http://www.computerworld.com/s/article/9243396/Healthcare.gov_website_didn_t_have_a_chance_in_hell_ http://www.computerworld.com/s/article/9243396/Healthcare.go.... Most startups fail. Most internal prototypes never amount to anything. This is just how engineering works.
- tootie 12y agoLater on the article describes the project as being "adrift" with ill-defined requirements and scope. That will kill absolutely any project and is a completely separate problem from technology. Lockheed failed at the engagement level to define scope. Their development team didn't have a prayer.
- phaus 12y ago
- discardorama 12y agoThe sad thing is not that the government pissed away $300M on this project; but that the prime, LMFS, will continue to get away with it because there are no penalties! But the system is rigged that way: if you do decide to hold them accountable, you'll find that they have documented every little specification change that the SSA put in after the project was awarded. So now the blame comes back to the SSA. Both sides will point fingers at each other; the consultants on this project will laugh (on their way to the bank), and things will remain the same. In other words: the system of contracting in place today is broken. From labyrinthine requirements to bid for one, to obtuse specifications to lack of oversight; it's one huge mess. And the system is corrupt to boot. Primes like LMFS, etc. employ former employees of these agencies (and vice versa), so it's very hard for an outsider to break in.
- easytiger 12y agoPennies in an ocean: http://www.theguardian.com/commentisfree/2011/aug/03/nhs-database-digital-disaster http://www.theguardian.com/commentisfree/2011/aug/03/nhs-dat... > ...final cost to be as high as £20bn, indicating a cost overrun of 440% to 770%
- Xophmeister 12y agoRather offtopic, but does anyone else find the dollar glyph in the header font jarring?
- quux 12y agoNow that you mention it, yeah it is... It looks like a lot of the symbols and punctuation characters in Garamond Pro are slanted with no logical pattern or reason. Would drive me nuts if I were using it. http://i.imgur.com/Cbd74Gq.png http://i.imgur.com/Cbd74Gq.png
- bane 12y agoFor more like this https://en.wikipedia.org/wiki/Virtual_Case_File https://en.wikipedia.org/wiki/Virtual_Case_File http://www.reuters.com/article/2012/03/14/us-manhattan-saic-idUSBRE82D14B20120314 http://www.reuters.com/article/2012/03/14/us-manhattan-saic-...
- mathattack 12y ago"For past 5 years, Release 1.0 is consistently projected to be 24-32 months away." I have worked on project like this. Usually it's the sign of management failure (poor communication, poorly defined requirements, bad governance, inability to retain knowledgable employees) rather than technical feasibility. It's not like the software is pushing technical boundaries. "Echoing McKinsey, Gershman said that without a strong leadership element, IT projects of this magnitude are almost doomed to fail from the start." This is a big problem. Because employees know these problems will fail, the smartest avoid them like the plague.
- remon 12y agoIt's amazing to me how consistently government IT projects fail and how widespread the problem is. A simple google teaches us that this has been and still is a consistent problem for quite a lot of countries. Some with an incredibly strong tech sector (the US is obviously a prime example). Surely there must be a way to organize/orchestrate government IT projects in a way that improves delivery and risk management. And...300 million USD?
- al2o3cr 12y agoScratch out "government" above - the overall failure rate of IT projects in general is crazy. The difference is that private project failures don't get news coverage (barring utter disasters that bankrupt the company).
- ryandvm 12y agoThis is business as usual in government contracting. The goal is not really to deliver working software - instead, it's multiple companies working together to figure out how they can maximally extract income from an entity with unfathomably deep pockets. It's layer upon layer of red tape sprinkled with the occasional sociopath just to spice things up. The filthiest part is that nobody but the taxpayer is interested in things being more efficient. The government agencies don't really care because there is no real financial accountability. Besides, if they use up all their budgets, they just get to ask for more next year. And contractors generally have their profits limited to a percentage of the project cost, so their goal is to balloon the cost as large as possible. For several years I worked on a very large government project. When you're sitting in a two-hour, 100 person, sprint review every two weeks and everyone is fucking off on their phones the whole time, it literally makes you question your sanity.
- steven2012 12y agoFeeding from the government trough is where people REALLY get rich. This is why Palantir is pretty successful, because getting in bed with the government means being able to feed off our tax money much easier at elevated prices.
- duaneb 12y ago> This is why Palantir is pretty successful Ahhhh
- 8102460 12y agoI have actually worked on the DCPS product as a former LM employee, so here's what I saw: there are two sides to the problem. The biggest problem is the number of stakeholders. Currently, each state has their own system to process disability cases, and each system has its own way of doing things. SSA & Lockheed are trying to consolidate these 50+ systems into one national system. Every state this product is rolled out in creates some new challenge to make it work the way the state's SSA office expects. Hopefully this new exec will have the power to make some decisions and say "no" once in awhile. Lockheed has their own set of problems, mostly hinging on the contract they've signed with SSA. They are forced to hire a certain number of subs per their contract - hiring a percentage of subs is not unusual, but the level in this contract is very high. As such, the subs know they can force feed trash into LM. The resumes coming from the subs were complete bullshit. Things like "10 years of experience administering X", but then in the interview, you find out they know next to nothing. Especially bad with databases when you even try to ask them how to write a join. One meeting we had while I was still there was a "skills assessment" - from 1 to 10, how well do you know X? There was no negative to this meeting, we just wanted to know who knew what. The number of "10s" in that meeting was beyond belief. So sure, Lockheed may not have a lot of experience here, but that problem is exacerbated by the stakeholders and the contractors they are forced to hire.
- nazgul 12y agoWhy are the forced to hire subcontractors, and a certain # of them? Do you know what the thinking is behind that?
- 8102460 12y agoThe overall idea behind being "forced" to hire subs is that it helps promote smaller, local businesses. In the past this worked out very well, because the subs brought in some very knowledgeable people. I believe the old number was 25% of the people, and you could pull from anywhere. If you had a 20 person team, you could have 15 LM, 2 from Company X, and 3 from Company Y. On DCPS, the problem is that LM is forced to hire 45% (FORTY-FIVE) of the subs through one specific subcontractor. That company had a previous contract with SSA, and SSA liked them. Since the sub essentially lost their work with SSA to LM, and they know LM was contractually obligated to hire through them, they stuffed us with a ton of below average folks. On the face of it, I agree with the idea of a large contractor having to sub out some of the work to the smaller players; it's just that LM overplayed their hand trying to get this contract, and got screwed for it. But that's another story on how LM management is trying to make revenue numbers for bonuses...
- androtheos 12y agoNo wonder taxes are so unbelievably high. The FBI did the same thing a few years ago. http://www.cnn.com/2005/US/02/03/fbi.computers/ http://www.cnn.com/2005/US/02/03/fbi.computers/
- duaneb 12y agoYes, it's not at all programs like the absolutely-crucial F-35 program costing $399B up front.
- junto 12y agoMcKinsey and Co probably charged the government a couple of million dollars to get some of its junior partners to write that report which eventually a senior partner signed off before golf and took the credit. Says the cynic in me...