5 ms·
> Actually testing was built into the software. When it came up it would talk to the physical parts to make sure everything was communicating ok before it could
by outworlder 11y ago
> Actually testing was built into the software. When it came up it would talk to the physical parts to make sure everything was communicating ok before it could start running.
I've been wondering about this for a while. We tend to run unit tests, integration tests, whatever tests, while the software is in development. However, once it is in "production" (for whatever definition of production), usually no tests are performed. At most, there's some sort of metrics and server monitoring, but nothing in the actual software.
It's a work of fiction, but Star Trek has these "diagnostics" that are run, with several levels of correctness checks, for pretty much everything. In your typical app, it could be useful to ask it to "run diagnostics", to see if everything was still performing as expected.
- jerf 11y agoIt doesn't necessarily take much work to run integration tests as diagnostics on production, especially if you plan on it in advance. I've had good success with it. Of course there's a certain level of destructive testing you can't do live, but that you really ought to do on your development system, load testing being a simple example. It behooves the wise developer to keep these quite separated in the code. :)
- rzzzt 11y agoDropwizard (a web application framework/library) incorporates the idea of such health checks, and you can also implement additional ones specific to the application. It is encouraged to run them periodically in production to ensure that the database connection is still up, threads are not stepping on each others' toes, etc. https://dropwizard.github.io/dropwizard/manual/core.html#health-checks https://dropwizard.github.io/dropwizard/manual/core.html#hea...
- joshrotenberg 11y agoThat uses the metrics library by the same author. Very handy stuff in there for development and production. https://dropwizard.github.io/metrics/3.1.0/ https://dropwizard.github.io/metrics/3.1.0/
- acomjean 11y agoEven while running the radar would keep track of communications between the parts and make sure things were still ok. The system needed messages periodically from the external components and vice versa to make sure things were ok. There were status messages sent around too. And a display of how things were doing. Its been a while, but I remember some of the things. You could command the external things to run diagnostics and report back. Part of this was it makes fixing things easier (like your car computer's diagnostics), part of it was required so we can figure out where things weren't working optimally. For example if a motor was running its controller computer didn't hear from the main system in X seconds, it would just stop, send a message about what it was doing and then wait for an instruction. Presumably this was to prevent all heck from breaking loose if the main system went down or was not responsive. This was all spelled out in long requirements documents. I wonder sometime if our cars controlling computers are doing this...
- mlonkibjuyhv 11y agoCars often disable any system capable of interfering with the wheels at the least sign of issue. Insufficient seal on the fuel-cap? Disable ABS, TCS, etc.
- nradov 11y agoThat's not correct. You can leave the fuel cap off and it won't disable ABS or TCS. The only thing that will happen is the "check engine" warning light will come on.
- acranox 11y agoThat's not correct. Cars will generally have something that's often called a 'limp home' mode. I saw a car that's PCM (powertrain control module) that failed some internal test, so it fell back to a basic mode where the engine wouldn't rev over about 2000RPM, and I'm sure all the emissions systems went into a basic fail-safe mode, where things like the fuel delivery goes into a hard-coded mode, instead of using feedback from the oxygen sensors to tune the fuel delivery. But a loose fuel cap doesn't disable the ABS or TCS. Even a single faulty wheel speed sensor doesn't have to disable the entire ABS, it can still independently assess each wheel, determine if a lock-up is imminent and modulate the brakes for that wheel.
- Tloewald 11y agoThe project I'm working on (for the Federal government no less, and very much not life and death) involves having test data in the production system and tests that run in the production system. (I was impressed by this when I found out.)
- clearf 11y agoMake sure they're well separated! c.f. http://www.wsj.com/articles/SB10001424052748703376504575491971336921954 http://www.wsj.com/articles/SB100014240527487033765045754919...
- Tloewald 11y agoThe stuff we are doing is so not life and death that nothing like this could happen. E.g. We have fairly insane levels of security for information that is inherently public.
- omouse 11y agoThat would be called reliability testing and it's why Netflix has chaos monkey to throw a wrench into things.
- lostcolony 11y agoAlso conformity monkey.
- dugmartin 11y agoEarly in my career I worked on military flight data recorders, including the development of the software for the F-22's "black box". Those systems have SBIT, IBIT, PBIT and MBIT sub-systems were BIT is "built in test" and S = startup, I = initiated, P = periodic and M = maintenance. I remember making the Star Trek diagnostic joke myself when I was assigned the SBIT work. Each BIT does varying level of testing based on it's runtime budget but there are a lot of very basic tests that don't make much sense until you see your first field report of some register bit "sticking". Its much better to ground a plane that can't add 1+1 than to find that out in mid-flight.
- Serow225 11y agoYup, as far as I remember all the avionics hardware I saw had these types of BITs built in.
- neurotech1 11y agoAn F-22 had a FCS failure and didn't realize takeoff. The pilot didn't do a IBIT on the FCS as required, therefore wasn't aware that all 3 rate sensors latched up. The jet was uncontrollable once it left the ground, and the pilot ejected safely. [0] As I recall, they modified the sensors to avoid latch up (extra pullup resister) and updated the FCS software to provide a warning if all 3 sensors return zero output. Even though this wasn't really an error in the FCS software, It could be argued that it failed to detect an erroneous input from 3 redundant sensors that latched up for the same reason. This is a classic example of a pilot who misunderstood the pre-takeoff checklist procedure and cost the USAF a $325m aircraft. [0] http://usaf.aib.law.af.mil/ExecSum2005/F-22A_20Dec04.pdf http://usaf.aib.law.af.mil/ExecSum2005/F-22A_20Dec04.pdf
- infinotize 11y agoTo be fair this seems to be more of a classic example of a documentation or training problem. Right from your link: > During the mishap sequence, the MP started engines, perfomled an IBIT, and had a fully functioning Flight Control System. Subsequently, the MP shut down engines to allow maintenance personnel to service the Stored Energy System. During engine shut down, the MA's Auxiliary Power System (APU) was running. The MP believed the APU provided continuous power to the Flight Control System, and therefore another IBIT after engine restart was unnecessary. This belief was based on academic training, technical data system description, and was shared by most F/A-22 personnel interviewed during the investigation.
- deleted 11y ago[deleted]
- grayarea 11y agoI used to write fire control and fire monitoring software. The whole idea of live testing is built into the ethos of such systems. In fire alarms and control the only tests that matter are the ones performed in a production environment.