7 ms·
> the navigational software on the $3B warship is far less capable any maps app on your phone. The software, called "Voyage Management System" (VMS), is the hub
by 2OEH8eoCRo0 5y ago
> the navigational software on the $3B warship is far less capable any maps app on your phone. The software, called "Voyage Management System" (VMS), is the hub of all the ship's planning. But it can't even do basic safety-of-ship alerting.
My phone can do "safety-of-ship" alerting? I'm the first to shit on defense software, it's my background, but map software for a submarine is safety-critical. While submerged it's basically their eyes. It has to be more reliable and trustworthy than possibly any other map software. Not only this but navigation information would have to be fused together from GPS as well as INS (inertial nav).
My 2 cents. Background in defense but not submarines.
- 0xfeba 5y agoI've worked in defense as well. Older software seemed to be tested more, or otherwise more reliable. Sure, it could be outdated, but generally it was usable. As they moved away from embedded to more networked and newer tech stacks things got terrible. The software world of today is not compatible with the needs of the defense industry. Plus all the red tape and misdirection. The software need comes from an organizational budget that has to use money to justify receiving money. Then it funnels down into political buckets, who use the dollar amount given to determine needs/wants. Then it goes back up the chain to the government. Then to a contractor, then a flurry of subcontractors. Then finally a developer. Rarely is the warfighter who will actually use the system consulted. Even then, rarely will the programmer meet that person using the software. Yet they all call themselves "agile" now.
- jwithington 5y agoI don't think all of those problems are unique to government software. Some (a lot?) enterprise software is atrocious and disliked by end users. That's because the people making the purchasing decisions != end users. But it's flabbergasting just how bad VMS is considering it's the navigation system for these national assets. It's not like an HR system with poor UX--navigation is essential to the safety and effectiveness of the whole platform!
- wolverine876 5y ago> Some (a lot?) enterprise software is atrocious and disliked by end users. That's because the people making the purchasing decisions != end users. It's because the people making decisions care about putting money where it earns the most return and have more on their plate than they can address. Making software more pleasant for internal users rarely makes it to the top of the list. > navigation is essential to the safety and effectiveness of the whole platform! Subs crash very rarely, so the software seems to be sufficient.
- jkhdigital 5y ago> Subs crash very rarely, so the software seems to be sufficient. Subs also operated for many decades without any kind of computer-assisted navigation, and training to handle the loss of such systems is routine, so your statement is pretty vacuous. The software at a minimum should allow sailors to operate as safely as they did without it, but one would assume that the entire point of software assistance is to improve safety and/or operational effectiveness.
- dn3500 5y agoI was a developer at a large contracter doing work for the Navy in the 1970s. Compared to today, there was almost zero code review, but an insane amount of testing, from the lowest module level to the highest integration level, both against the requirements and in simulation. We did not subcontract software. I often wished I could talk to the warfighter who was going to use my stuff but it never happened. Worse I never got any feedback on whether my stuff was even used. We did have quite a few ex-Navy people in our dev group and that did help.
- wolverine876 5y agoHow efficient was that? How fast could you develop solutions?
- virtue3 5y agohttps://news.usni.org/2019/08/09/navy-reverting-ddgs-back-to-physical-throttles-after-fleet-rejects-touchscreen-controls https://news.usni.org/2019/08/09/navy-reverting-ddgs-back-to... Fucking bad from what I've read. It's a really bad problem still and the USN needs to correct it by having a tighter integration with the HW/SW people and the actual soldiers/sailors.
- dkdk8283 5y agoHopefully very slowly, you do not want fast development. You’re developing a safety critical system and should prioritize stability over features.
- dn3500 5y agoIn a way it was awful. It was sort of like anti-agile. There was absolutely no dialog between the devs and the customer. If the requirements said the throttle had to be on the touchscreen (to use an example from another comment), that's what we did, even if we strongly suspected this was a really bad idea. I was too far down the food chain to know where these requirements came from, it was like they were handed down to Moses on stone tablets. How fast could we develop? In addition to the requirements, we were also handed a schedule (a PERT chart, if anyone still knows what that is). It was laughably padded. Padded so much that it would have been impossible to fall behind schedule. I almost always finished my work for the day before lunch. Sometimes I would then go for a three martini lunch with the sales guys, or spend the afternoon in the library or flying the F-14 simulator. Sometimes I would spend the afternoon working on tools to make debugging easier. It was a fun job, but ultimately unsatisfying and I left after two years.
- VBprogrammer 5y agoI suspect any kind of sustained warfare in the modern era would result in the immediate obsolescence of much of the equipment designed in peacetime.
- noja 5y agoWarfare would not even be required https://en.wikipedia.org/wiki/Carrington_Event https://en.wikipedia.org/wiki/Carrington_Event
- jcadam 5y agoThe only time I had contact with the end user as a defense contractor was when I was working on-site in an R&D shop, where we were creating software for users that were all in the same "integrated" team with us. That was the only job I've had in the industry I could describe as "fast-paced."
- zentiggr 5y agoTwenty years ago, when VMS just got introduced, it behaved like this... I can't believe that after all this time, it's still completely borked, in all the same ways. Somebody at NAVSEA needs to be dragged by their nostrils out on a deployment, take notes, and go back to their office with their pride in a garbage bag. Proud as hell to have served, and angry as hell that our current crews are still dealing with the same ---procurement--- failures.
- Jtsummers 5y agoThat's from the time when everything was still (and everything still being planned for the replacements were also) big-bang releases. 5-10 year projects to replace the broken system that still lack the necessary capabilities. I've never worked on a Navy project, but I've seen several spectacular USAF failures of similar magnitude. The most hilarious (in an absurdist sense) was having 3 generations of a system running concurrently because none of them had all the necessary features, when I left they were working on number 4.
- Goety 5y agoWell... What is your solution for this?
- rbanffy 5y agoMine would be shorter development cycles and incremental deployment of improvements as soon as basic functionality is achieved. Work in an accurate simulator with an experienced crew from day one would also help.
- Jtsummers 5y agoDon't plan projects out 5+ years and fail to revisit the plan. That is, don't do Waterfall. Work with the actual user/operators (this I did see a lot of, but not on new systems, only on systems in "maintenance") to get regular feedback. One of the best jobs of this I saw brought in both current and retired operators. Current ones rotated through helping evaluate the system and the requirements, retired ones were hired on to be part of test/QA. Switch from fixed contracts to more level of effort contracts. Generally shift towards continuous delivery where possible (with information systems much easier than with embedded systems, but doable to an extent with embedded systems). Fix the official release test phase which is often 6-24 months at the end, most of it waiting. This is part of the program's overall process which is very Waterfall in structure for most systems. I've seen work that was completed and put on a shelf for over a year before being evaluated for flight readiness. Well, it was useless to operators by that point [0] and delayed vital feedback for the next iteration. That's partly a logistics problem, but addressable. Fix the lack of trust the operational test teams have for development teams. There was a complete lack of trust for the teams delivering the product to be tested, but the test plans for flight testing were often so bad that they were a joke (for the particular delivered systems, not flight test plans in general). If you don't have the trust, gain the trust. Take the existing operational test plans and back port them to your own test infrastructure and test efforts. Deliver that report along with the product, eventually they may trust your test reports and can start speeding things along. As with all software, get as much testing done as early as you can. The longer you wait, the more costly it is to address discovered issues, particularly validation issues. Finally, get experienced software engineers into program offices. Most of the time their "software engineers" are a recent college graduate, or someone from a totally unrelated field that wrote some Matlab code once. They lack sufficient experience to properly manage or guide the management of the software portion of a project. The worst I saw with this was that system I mentioned in my previous comment, the program office lead for the software portion was a newly pinned on 1st LT history major. Zero technical experience (had been in some finance role before), they trusted the contractors too much and made decisions more on their input than anyone actually using the system. [0] For context, this is software that spent 2-4 years being developed and then would be shelved for 1-2 years before being tested. Which meant it still had 1-2 years for fielding. So from conception to release you're talking 4-8 years. That is an awful pace.
- denkmoon 5y agoAs a defence contractor I have implemented many incredibly stupid or pointless things merely to fulfill someone's idea of "agile". Fortunately all the projects I have worked on aren't related to combat or anything where actual people could be hurt.
- LeifCarrotson 5y agoYour phone has Google Maps or Apple Maps, which will alert you when there are road closures or hazards ahead, yes. It even has a list of places where there are roads, and places where there are not roads, and will not (typically) direct you to go off-roading. The behavior of VMS is like punching in a street address and having Maps tell you to go straight there as the crow flies. It's hard to redundantly guarantee correctness, if you wanted to rely on VMS to avoid hitting the seafloor that would indeed be difficult. But come on, how hard is it to offer a highlight for the shallowest points near your course, or a warning that says "you added an extra zero here"?
- jjk166 5y ago> Your phone has Google Maps or Apple Maps, which will alert you when there are road closures or hazards ahead, yes. It even has a list of places where there are roads, and places where there are not roads, and will not (typically) direct you to go off-roading. Google's servers have this information, compiled in real time from countless sources including direct satellite cartography and all beamed to a phone acting as little more than a user interface. Take your phone a few hundred feet below the water where it is cut off from the outside world and even the best maps in the world aren't accurate and you're going to run into a lot of things.
- trcarney 5y agoYou could have a local copy onboard. There should be enough rack space on the boat to keep a copy that gets updated when the boat gets connected to shore data. Also for sea data, the data set would be much smaller because it hardly changes, and you don't have as many variables(i.e. live traffic, construction, etc), especially under the sea.
- jjk166 5y agoI don't agree that the data set would be much smaller. Maps for driving may look complicated, but they're just weighted graphs. You might have a few different data points for MLK drive like length, average traffic, speed limit, class of road, etc, and these values might even be given for each short stretch of the road, but you don't have to have any knowledge of the actual topography, and all of these values can be safely taken as rough estimates. No one is going to die if you falsely classify a stretch of road as high traffic when it's really moderate. For a submarine, you need to know the topography of the sea floor down to meter scales. Accurate maps are few and far between, and difficult to create. Changing currents, underwater landslides, silt from rivers, sunken wreckage and even just garbage can create and move obstacles large enough for a submarine to care about in relatively short timeframes. Of course submarines can and do navigate with maps prepared ahead of time, but it's tough, and every now and then you get a run in with a sea mount.
- Waterluvian 5y agoExactly how I feel when someone shows up to the GIS domain with Google Maps and is like “why can’t you old timers be more like this?!”
- roywiggins 5y agoNavy software (among, to be fair, other things) killed 10 sailors in a preventable collision: https://features.propublica.org/navy-uss-mccain-crash/navy-installed-touch-screen-steering-ten-sailors-paid-with-their-lives/ https://features.propublica.org/navy-uss-mccain-crash/navy-i...
- KennyBlanken 5y agoLack of training killed them. The ship's command put a farm kid with barely four months in the navy at the controls of a warship in an area that is infamous for how congested and difficult it is to navigate. Not four months experience on the bridge. Four months from when he joined the navy. > Bordeaux felt confident using the system to control the speed and heading of the ship. But there were many things he did not understand about the array of dials, arrows and data that filled the touch screen. > “There was actually a lot of functions on there that I had no clue what on earth they did,” Bordeaux said of the system. As for his boss: > The chief petty officer had himself received less than an hour of instruction. That's the guy in charge of training the bridge staff. Nobody knew how to use the system.
- aaron695 5y ago> My phone can do "safety-of-ship" alerting? Yes. Never controlled a drone from your phone? The fact you've worked defence and can't see how consumer devices do things submarines can't is the issue. We all know a submarine is worth x billion, a drone much less. We understand about critical failure on each. You job is to work out how to merge the two, but instead defence hides behind ideas like verifying software. Software verification should be a tool, not a wall for bureaucrats to hide behind.
- mpyne 5y ago> It has to be more reliable and trustworthy than possibly any other map software. Did you read the detailed examples on the website? Because the point you make is the same point the website makes: VMS is crap, compared to the map application that comes with your phone by default. It is less reliable, because it crashes all the time. It is less trustworthy, because you can't even rely on it to do extremely basic checks on the safety of a proposed leg of your navigation plan. I find the website disappointing for a different reason: the flaws they point out in VMS in 2021 are the same flaws I experienced with VMS from 2006 to 2009 during my own time in submarines. I should be astonished that we haven't made things better since, but I've seen enough of software in defense to honestly claim surprise.
- samplatt 5y agoA $3B warship costs less than what (I imagine) has been spent on Google Maps development. I'm currently working on ship-alerting software, and all the usual "haha programmers suck at their job" memes apply... But then, so does the hardware it's running on. PLC/SCADA hardware that behaves unpredictably with zero support from the (global monopoly) vendor and PC's that are years out of date because getting new hardware approved is fucking torture.
- jkhdigital 5y agoMultiply that by ~70 (number of active subs in the fleet)
- mpyne 5y agoVMS is used across submarine designs spanning all the way back to 2005. Currently there are 67 submarines actively part of the battle fleet (https://www.nvr.navy.mil/NVRSHIPS/FLEETSIZE.HTML https://www.nvr.navy.mil/NVRSHIPS/FLEETSIZE.HTML), let's say $3B per submarine, that's $200 *BILLION* dollars of capital expenditure. And it should go without saying that the reason we spend money on submarines is because the missions they go on are that important to the national defense. I think Google has spent less than $200 billion on Google Maps. Between the capital expense and the mission to perform, and the repeated underground collisions, I don't think it's too much to ask that the Navy apply some usability design to support one of the most critical functions of any warship, safe navigation. Because I've used VMS myself and I can attest to everything the author of this article mentions, and it's far from being just a resourcing question: it's inherent to the way the Navy designs, implements and acquires software-based systems. This includes the stuff you mention about hardware (which, oddly, the Navy submarine force had managed to subvert with ARCI sonars, which use commercial servers and a ton of additional spares to meet operational availability needs).
- KennyBlanken 5y ago> My phone can do "safety-of-ship" alerting? Yes. It's a very similar problem to aviation terrain avoidance systems which are now so common, and such a "solved problem", that numerous popular smartphone apps for flying and gliding feature not only terrain warning but features like "will I be able to make it to that airport" determination (for gliders) which factors in a slew of real-time parameters and terrain data. And, really, "is the plane going to hit terrain the way it's being flown right now?" is far more complex than the problem the author was describing, which "is what you're planning to do going to hit terrain." They commanded a dive to a depth below the ocean floor, and it wasn't like they were trying to run a canyon like Hunt For the Red October...from the sounds of it, nothing in the area was deep enough to permit that dive to happen safely. This problem is also a problem the PC/console game development community solved, oh, around two decades ago. For quite some time games have simulated bullet physics while also determining if a shot hits a moving, deforming, 3d object, with checks happening in a matter of ~10ms or thereabouts. "Will the submarine pass within X feet of underwater terrain" is a walk in the park. There's also the matter of the system not needing to assure a maneuver is safe, but merely to serve as an alert when something is obviously not safe. Given how much these things cost, as you point out, I'd say such a system is beyond warranted. > Not only this but navigation information would have to be fused together from GPS as well as INS (inertial nav). I'm going to guess that the Navy has figured out how to very precisely geolocate their submarines and has beyond-excellent seafloor map data.
- ThinkBeat 5y agoWhat computer hardware is this running on? What sort of "server farm" does a modern submarine have? Do they need to be super hardened operation in a vehicle that may be called to launch its missiles in a high radiation zone? In a worst-case scenario the sub my surface in the middle of or end of a nuclear war. They probably want to ensure that the missiles get underway even in such extreme conditions
- jwithington 5y agoYes, your phone can alert you when you're traveling over the speed limit or if there are hazardous conditions on the road. VMS can't do anything like that.