35 ms·

Sounds like a load of crap to me. Seriously, 500 million lines of code? I bet not even mission-critical code written to power NASA spacecraft and satellites are

Sounds like a load of crap to me. Seriously, 500 million lines of code? I bet not even mission-critical code written to power NASA spacecraft and satellites are anywhere near that length. I propose they give the project to the HN community, divide up the $400 million figure amongst the participants and give us a weekend and I bet we'll have more to show than that of the so-called developers the government hired. I am not underestimating the complexity of such a site, but seriously, it's not like it hasn't been done elsewhere to a degree. if I can do Internet banking, the a site for providing healthcare can be built a lot easier. They're both of similar scale and security requirements I would imagine. This site seems like the greatest scam ever perpetuated. I bet the contractors are laughing themselves to the bank literally as we speak. Even though it's bad publicity, money is a higher priority than credibility for some businesses out there it would seem. Is it any wonder people don't trust the government? They can't even hire competent people to build them a website. This seems to be a recurring theme for local councils and governments elsewhere as well, so it's not just the US this is happening. People need to get mad about this. That's $400 million dollars that could have been invested elsewhere like medical research, charities, food programs, rehabilitation for the homeless... A $400 million mistake has been made, and another $400 million will probably be spent before it's even ready. What a waste of money.

javert 13y ago
While I completely agree with you in general, a nit pick: > I bet not even mission-critical code written to power NASA spacecraft and satellites are anywhere near that length. That doesn't seem like a good example. You want to keep the code in those kinds of things to a minimum, and that's probably not too hard.
- DigitalSea 13y ago
  Yeah, that's the best example I could come up with at the time of writing the comment. It's been a long day today, and I don't mind the nit-pick, you are right.
- code_duck 13y ago
  It's best to keep the code in everything to the minimum!
  - derleth 13y ago
    > It's best to keep the code in everything to the minimum! Optimizing for readability is usually most important. Readability is usually more important than reusability, in fact, although readability very often enables reusability. But we're talking about mission-critical embedded code, which is a special case in many respects. Still, making it as readable as possible makes bugs easier to find.
    - EdSharkey 13y ago
      I prefer the term "maintainability" to "readability", and I completely agree with you there. Reuse is an emergent property of code, though. It's very tough to engineer for reuse upfront and have that effort be successful. It's best to keep your code simple and put your effort into making it well-tested. Then, over time spot the commonalities in everyone's code, and factor out the common bits to become reusable code if everyone agrees it's worth it to do so.
    - javert 13y ago
      Thanks for saying this... it needs to be said far more often. I wish there were a book about how to READ software. (If anyone knows of one, please speak up.) One tip I picked up on my own over time is when you're looking at an unfamiliar project, don't read the code like a book, read it like a program. In other words, try to find main() and trace how things work from the bottom up, rather than trying to understand it based on how the authors have broken up the code into directories. I don't know yet whether this is truly good advice.
    - code_duck 13y ago
      I definitely agree with that. I was thinking of minimal code as a way to achieve the ends of readability and maintainability. Excessive minimalism goes against both of those goals, when you get to compactness for the sake of compactness, perl code golf or micro optimization style.
- keypusher 13y ago
  Yeah, I remember reading somewhere that they have extremely strict standards at NASA, to the point that multiple reviews are mandatory on even single line changes for mission critical systems. Here's an example: http://spinroot.com/gerard/pdf/P10.pdf http://spinroot.com/gerard/pdf/P10.pdf And another: http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf Dunno how closely these are followed in day-to-day operations, but they clearly put some thought into it.
EdSharkey 13y ago
Yes, we've burned at least $400 mil. Last week, everyone insisted it only cost $55 million, and the $768 million figure was misreported. I can only speculate what the final cost will be after victory is declared over these bugs. It is too late to get mad about any new/rising costs related to the ACA. It is here to stay and might as well be Medicare 2. "Law of the land" as the president is fond of saying. Here's a fascinating visio diagram that shows just how much government has grown under the ACA. Anything red indicates new or expanded government: http://www.jec.senate.gov/republicans/public/index.cfm?a=Files.Serve&File_id=8e6dbf03-ca4a-44be-9de4-a100c43fb5c8 http://www.jec.senate.gov/republicans/public/index.cfm?a=Fil... Seems like the ACA created very painful integrations between federal systems and insurers, asking them to do many new things. Not a big surprise they're having problems. Some federal IT systems are archaic and integrations/change are costly and fraught with danger. The insider stories I've read about the Soc Security mainframe-based system are frightening!
- EdSharkey 13y ago
  Here's the frightening story I recalled on Social Security from 2011. http://www.informationweek.com/government/enterprise-architecture/social-security-confronts-it-obsolescenc/229300595 http://www.informationweek.com/government/enterprise-archite... Woe be to the phalanx of ACA consultants that has to integrate with those systems... :/
- hga 13y ago
  "It is too late to get mad about any new/rising costs related to the ACA. It is here to stay and might as well be Medicare 2. "Law of the land" as the president is fond of saying." Reading this article, which at the highest level tells us the people in charge, the White House and HSS's CMMS (all government workers, the latter the integrator of the project) are still discussing how to fix this mess, hoping to have a plan by Thursday, combined with e.g. CMMS delivering changes to the contractors through the week before launch, and per this article "In the last 10 months alone, government documents show, officials modified hardware and software requirements for the exchange seven times.", suggests to me it could easily be years before the exchange works, and possibly never. We've seen this before (FAA, FBI, IRS, etc.). All depends on CMMS (non-specialist civil servants and perhaps political appointees) learning big project management "on the job" or the vast majority of them getting replaced by an integrator who has a clue, either of which will take months. That's just what's required to get sane high level management of the project. In the meanwhile, can the individual parts done by different contractors get some serious progress? Well, first it depends on CMMS freezing requirements, and doing a good enough job of deciding what requires a total rewrite. Per this article: "Quality Software Services Inc., or Q.S.S.I., a unit of the UnitedHealth Group, developed the identity management system, another major component that allowed consumers to register and establish accounts. The identity management system from Q.S.S.I., which also taps into government databases to retrieve users’ personal information, was a particular source of trouble when the exchange opened. Change orders show that on Oct. 4 — after millions of people had been trapped in technological loops trying merely to log in — the government asked CGI to help it devise a new identity management system to replace the one provided by Q.S.S.I. But specialists said that approach was abandoned as too risky. Ultimately it was decided to fix the current identity system." And we've heard from a HN member who was involved in a big university project that Oracle's identity software is fairly gnarly to get working. Which I have absolutely no trouble believing, because the underlying problem is gnarly. But one wonders just how much panic was behind CMMS's idea of punting the existing one 3 days after launch. And you know the managers of both contractors had to spend a lot of time working with CMMS on just this detail ... working with people who are simply not (very) experienced in software development and management. (Ask for my worst experience working for a government for the level of cluelessness that's likely involved here.) So, getting to my point, you may say "It is here to stay", but "You Can't Fight Mother Nature" or the hard reality that computers do exactly what you tell them to do, not what you want them to do. How many months or years do you think people will stand for being fined by the IRS for not enrolling when they can't? How many will die because they've lost their existing coverage due to the ACA changes but can't sign up for anything, or anything they can afford without the subsidies which only the exchanges can (now) deliver? What happens if this not entirely nonfunctional system develops an adverse selection death spiral? That's where only the very dedicated, the ones who really need a lot of healthcare, jump through the hoops, whereas the healthy 20 somethings and beyond, who are asked to do a 30 step (!) application on an unreliable system, punt, pay the fine but not the much larger sums needed to subsidize the 50-64 age bracket. Which raises the prices of the policies offered, which further decreases enrollment....

javert 13y ago

While I completely agree with you in general, a nit pick: > I bet not even mission-critical code written to power NASA spacecraft and satellites are anywhere near that length. That doesn't seem like a good example. You want to keep the code in those kinds of things to a minimum, and that's probably not too hard.

DigitalSea 13y ago
Yeah, that's the best example I could come up with at the time of writing the comment. It's been a long day today, and I don't mind the nit-pick, you are right.
code_duck 13y ago
It's best to keep the code in everything to the minimum!
- derleth 13y ago
  > It's best to keep the code in everything to the minimum! Optimizing for readability is usually most important. Readability is usually more important than reusability, in fact, although readability very often enables reusability. But we're talking about mission-critical embedded code, which is a special case in many respects. Still, making it as readable as possible makes bugs easier to find.
  - EdSharkey 13y ago
    I prefer the term "maintainability" to "readability", and I completely agree with you there. Reuse is an emergent property of code, though. It's very tough to engineer for reuse upfront and have that effort be successful. It's best to keep your code simple and put your effort into making it well-tested. Then, over time spot the commonalities in everyone's code, and factor out the common bits to become reusable code if everyone agrees it's worth it to do so.
  - javert 13y ago
    Thanks for saying this... it needs to be said far more often. I wish there were a book about how to READ software. (If anyone knows of one, please speak up.) One tip I picked up on my own over time is when you're looking at an unfamiliar project, don't read the code like a book, read it like a program. In other words, try to find main() and trace how things work from the bottom up, rather than trying to understand it based on how the authors have broken up the code into directories. I don't know yet whether this is truly good advice.
  - code_duck 13y ago
    I definitely agree with that. I was thinking of minimal code as a way to achieve the ends of readability and maintainability. Excessive minimalism goes against both of those goals, when you get to compactness for the sake of compactness, perl code golf or micro optimization style.
keypusher 13y ago
Yeah, I remember reading somewhere that they have extremely strict standards at NASA, to the point that multiple reviews are mandatory on even single line changes for mission critical systems. Here's an example: http://spinroot.com/gerard/pdf/P10.pdf http://spinroot.com/gerard/pdf/P10.pdf And another: http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf Dunno how closely these are followed in day-to-day operations, but they clearly put some thought into it.

EdSharkey 13y ago

Yes, we've burned at least $400 mil. Last week, everyone insisted it only cost $55 million, and the $768 million figure was misreported. I can only speculate what the final cost will be after victory is declared over these bugs. It is too late to get mad about any new/rising costs related to the ACA. It is here to stay and might as well be Medicare 2. "Law of the land" as the president is fond of saying. Here's a fascinating visio diagram that shows just how much government has grown under the ACA. Anything red indicates new or expanded government: http://www.jec.senate.gov/republicans/public/index.cfm?a=Files.Serve&File_id=8e6dbf03-ca4a-44be-9de4-a100c43fb5c8 http://www.jec.senate.gov/republicans/public/index.cfm?a=Fil... Seems like the ACA created very painful integrations between federal systems and insurers, asking them to do many new things. Not a big surprise they're having problems. Some federal IT systems are archaic and integrations/change are costly and fraught with danger. The insider stories I've read about the Soc Security mainframe-based system are frightening!

EdSharkey 13y ago
Here's the frightening story I recalled on Social Security from 2011. http://www.informationweek.com/government/enterprise-architecture/social-security-confronts-it-obsolescenc/229300595 http://www.informationweek.com/government/enterprise-archite... Woe be to the phalanx of ACA consultants that has to integrate with those systems... :/
hga 13y ago
"It is too late to get mad about any new/rising costs related to the ACA. It is here to stay and might as well be Medicare 2. "Law of the land" as the president is fond of saying." Reading this article, which at the highest level tells us the people in charge, the White House and HSS's CMMS (all government workers, the latter the integrator of the project) are still discussing how to fix this mess, hoping to have a plan by Thursday, combined with e.g. CMMS delivering changes to the contractors through the week before launch, and per this article "In the last 10 months alone, government documents show, officials modified hardware and software requirements for the exchange seven times.", suggests to me it could easily be years before the exchange works, and possibly never. We've seen this before (FAA, FBI, IRS, etc.). All depends on CMMS (non-specialist civil servants and perhaps political appointees) learning big project management "on the job" or the vast majority of them getting replaced by an integrator who has a clue, either of which will take months. That's just what's required to get sane high level management of the project. In the meanwhile, can the individual parts done by different contractors get some serious progress? Well, first it depends on CMMS freezing requirements, and doing a good enough job of deciding what requires a total rewrite. Per this article: "Quality Software Services Inc., or Q.S.S.I., a unit of the UnitedHealth Group, developed the identity management system, another major component that allowed consumers to register and establish accounts. The identity management system from Q.S.S.I., which also taps into government databases to retrieve users’ personal information, was a particular source of trouble when the exchange opened. Change orders show that on Oct. 4 — after millions of people had been trapped in technological loops trying merely to log in — the government asked CGI to help it devise a new identity management system to replace the one provided by Q.S.S.I. But specialists said that approach was abandoned as too risky. Ultimately it was decided to fix the current identity system." And we've heard from a HN member who was involved in a big university project that Oracle's identity software is fairly gnarly to get working. Which I have absolutely no trouble believing, because the underlying problem is gnarly. But one wonders just how much panic was behind CMMS's idea of punting the existing one 3 days after launch. And you know the managers of both contractors had to spend a lot of time working with CMMS on just this detail ... working with people who are simply not (very) experienced in software development and management. (Ask for my worst experience working for a government for the level of cluelessness that's likely involved here.) So, getting to my point, you may say "It is here to stay", but "You Can't Fight Mother Nature" or the hard reality that computers do exactly what you tell them to do, not what you want them to do. How many months or years do you think people will stand for being fined by the IRS for not enrolling when they can't? How many will die because they've lost their existing coverage due to the ACA changes but can't sign up for anything, or anything they can afford without the subsidies which only the exchanges can (now) deliver? What happens if this not entirely nonfunctional system develops an adverse selection death spiral? That's where only the very dedicated, the ones who really need a lot of healthcare, jump through the hoops, whereas the healthy 20 somethings and beyond, who are asked to do a 30 step (!) application on an unreliable system, punt, pay the fine but not the much larger sums needed to subsidize the 50-64 age bracket. Which raises the prices of the policies offered, which further decreases enrollment....