9 ms·

Sounds kinda disappointing/irresponsible. Chromium is an overly complex, relatively buggy piece of software. For example I recently experienced a bug where a te

Sounds kinda disappointing/irresponsible. Chromium is an overly complex, relatively buggy piece of software. For example I recently experienced a bug where a text I selected was significantly offset from what should have been selected given the pointer position (on linux), a misclicky kind of bug. Considering that the majority of Crew Dragon's controls are through touch screen, maybe even manual docking(not sure, but I don't see a joystick anywhere), my personal experience with chromium does not instill confidence for such critical applications. But I guess they tested it very thoroughly. C++ and linux do not strike me as mission critical piece of software / language either. My heart desires for something with verified properties, advanced languages with integrated proof checkers, verified OSs, or at least smaller purpose built OSs that were carefully reviewed. But I guess SpaceX is not to blame for this. It's likely such approaches are simply not ready yet. And I know that C++ and linux are common in mission critical applications. JSF avionics is written in C++ for example. (Not to mention JavaScript in space, in a mission critical role. That shouldn't have happened.)

jchw 6y ago
Once you decide that absolute reliability is unattainable, you can instead aim for sufficient redundancy. It’s not perfect, but with enough safe guards in place it may very well be good enough.
- microcolonel 6y ago
  Software is not always like hardware though: the same software failure can cascade across redundant instances rapidly just as easily as it affects the primary.
  - jchw 6y ago
    The same hardware failure can cascade too; redundancy alone is not a panacea, you need some safeguards. Ideally, nothing should fully trust anything else.
    - microcolonel 6y ago
      Maybe cascade was the wrong word. I meant that if you pop in a hot spare and it's running the same software, there's a good chance it will encounter the same issue, sometimes immediately.
      - perl4ever 6y ago
        In some cases redundant software is developed independently by different teams to the same spec; I think I read about that with the Space Shuttle. How correlated are your layers of swiss cheese...?
    - perl4ever 6y ago
      I was just reading that they design the rocket so that one engine could explode without affecting the others (hopefully) and they test the engines by dropping a nut or bolt or something in, which is not something rocket engines normally can tolerate.
ryder9 6y ago
you sure showed those people who worked hundreds of hours and have more knowledge of their system than some rando in this 'intellectual' website
- 7leafer 6y ago
  you sure gave enough arguments to beat those of the parent.
  - ryder9 6y ago
    not on me to prove his arguments stable genius
davedx 6y ago
Why not Javascript? I would argue that a good usable UI in a spacecraft is a good thing, and Javascript is a great language for building UIs
- fdw 6y ago
  > Javascript is a great language for building UIs I disagree. It is easy to build something presentable with HTML and browsers, and there have been many JS frameworks that lower the barrier. However, that is not because JS is a good language for it; it is because JS is the only language available in that environment. In fact, there are at least two other languages that transpile to JS, because you need JS to be able to run in a browser. But JS the language is not good enough, so you want two write your code in something else. Building a UI in a completely different language, without a browser, is definitely harder, but you might have several advantages in the other, non-UI parts.
  - davedx 6y ago
    You haven’t made any arguments about why Javascript isn’t good for building UIs there. Here’s mine for why it is: - first class functions (callback style is a good fit for UI programming) - async/await and single threaded - many components of UIs are async by nature, Javascript supports you well here while keeping things simple with its single threaded model
    - vertex-four 6y ago
      There's... a large number of languages with both of those features.
      - scotty79 6y ago
        Could you name few better than js?
        vertex-four 6y ago
        In the context of writing reliable software for a spacecraft? Pretty much anything with a reasonable type system, for a start. Maybe C#, or some reasonable subset of C++20, or Rust. Maybe Reason or OCaml.
raverbashing 6y ago
I don't see why. Chromium is being used in the UI. Not on critical flight software. > maybe even manual docking It isn't done manually > C++ and linux do not strike me as mission critical piece of software / language either A lot of flight software is done on C++, as you even mentioned. In the end, your best reliability partner is not using a language that's only know by fewer people. What would be a good "mission critical piece of software"? QNX (with no memory protection?). Linux is fine. I suppose they're not firing Ubuntu out of the box and running that with no customizations. Separation of concerns and redundancy usually beat correctness on the long term (which doesn't mean they didn't test the correctness of the system).
- deleted 6y ago
  [deleted]
- StillBored 6y ago
  QNX (with no memory protection?) Its probably a misunderstanding, but QNX has a very strong memory protection model that extends to its driver stack. Given that its a true realtime/microkernel, which has its own set of problems, but memory protection isn't one of them. For a number of cases QNX is going to be a large step up from linux. http://www.qnx.com/developers/docs/6.5.0/topic/com.qnx.doc.ide.userguide/topic/memory_MEMMGT_.html http://www.qnx.com/developers/docs/6.5.0/topic/com.qnx.doc.i...
mofosyne 6y ago
Unlikely that they are using javascript in the actual flight controller. Also under the screen they have a small amount of manual buttons that is likely to be seperate from the touchscreen interface logic.
- nullifidian 6y ago
  >javascript in the actual flight controller. zero chances that it's the case. I meant that in aerospace applications UI is mission critical too.
  - eqvinox 6y ago
    > I meant that in aerospace applications UI is mission critical too. It is, but usability of the UI is mission critical as well. The humans in the capsule need to be able to quickly and effectively consume information in a way that doesn't make important facts hard to get at, and then quickly and effectively perform commands to address issues. I don't think a touch GUI is a bad choice in this regard, and if I'm building a touch GUI I'd rather use some existing framework. My personal choice would've been something like Gtk or Qt though :)
elteto 6y ago
In this case, C++ code is written in a different style than regular applications: exceptions are not allowed, allocations are not allowed past a certain point in program initialization, raw pointers are frowned upon, etc. This is common in this type of software and when you pair it with the right development culture it really makes a difference in the robustness of the result.
- justinclift 6y ago
  Any idea if there's a database layer (eg SQLite, PG, etc) on the systems?
  - elteto 6y ago
    On the ground side, yes, there are many databases for different purposes. But not on the actual flight software.
    - exikyut 6y ago
      Very curious, how is state persisted where non-volatile storage is needed, and how is that persistence structured?
      - Nicksil 6y ago
        State is, as far as the system is concerned, is what values the multitude of sensors are reporting at any given moment.
      - elteto 6y ago
        It is never persisted outside of RAM. The flight computers are never powered off during a mission. You rely on two flight computers to always be functioning, that’s why the system is one-fault redundant. You can think as each string being “functional“: given the same set of inputs you expect the same set of outputs.
        6y ago
eqvinox 6y ago
I'm reasonably sure that quite a bit of the complexity is more or less directly caused by the amount of features it provides. I agree with you in that Chromium probably brings in a lot of features they don't need, but I would also say that reimplementing the features they do need is also a huge liability and would incur quite a bit of complexity too... but it'd make it their insular system with no other resources flowing into improving it. I don't have the data to make that call, but I would think someone at their end evaluated this quite thoroughly and determined the risks of rolling their own to exceed the risks of using Chromium. (Also they mentioned on the stream that they do in fact have a few hardware buttons for important things. Can't cite this, sorry...)
stefan_ 6y ago
The fact that it's implemented in JavaScript and Chromium just tells you the fancy display is entirely superfluous to mission success.
- htk 6y ago
  Interesting way to see it. For anyone wondering how unreliable it is, think instead that the trade off was well planned.
- yellowapple 6y ago
  Is there a backup should that fancy display fail? If not, then I'd hardly call it "superfluous to mission success".
  - Czarcasm 6y ago
    Apparently the physical controls below the screens can perform all of the mission critical functions in the event of a display failure.
    - _emacsomancer_ 6y ago
      If only browsers had a similar fallback.
vijaybritto 6y ago
It should be most likely be WebGL and some glue js code. I don't think they're running everything in just js for the view code.
Nokinside 6y ago
Most safety critical code in the world, both civilian and military, is currently written in C/C++. Ada/SPARK is also used but it 's below 50%. These languages have the best practical static code analyzer, verification and proofing tools money can by. I'm personally using Astree https://www.absint.com/astree/index.htm https://www.absint.com/astree/index.htm Automatic docking software for the ATV that delivers supply to ISS is written using C code and verified with Astree. People get stuck into the language, but if it has some features that make it work, you can use it with verifiers. Then you write tests and simulators and tests ... JavaScript and Chromium seems more suspect to me, but if the code is well tested and has limited run time, I think it can be good to go. The state is stored and prosessed in those radiation hardened RAD750 processors. Chromium/JavaScript runtime image can restart every few seconds if needed. The UI and controls seem like unnecessarily flashy bullshit to me. Usually if you diverge from KISS principle bad things happen. But before I make any judgements I would see how they made those decisions.
- thebruce87m 6y ago
  How much is Astrée? I get nervous when you can’t see a price on the website.
edsac_xyzw 6y ago
In the case of SpaceX, it seems that they use embedded Linux and C++ with x86 processors, same as standard PC processors. A reason for using embedded linux is that, it allows using standard C++ or even scripting languages for controlling the hardware from the user-space by just reading and writing files. Linux device drivers (aka kernel modules) maps the hardware to special files on /sys or /dev. For instance, it makes possible to control a GPIO (General Purpose IO) which the device driver maps to sysfs special file system, by just writing 1 or 0 to the file a like /sys/class/gpio/gpio4/value which would enable GPIO 4 and turn on a LED attached to it. Another practical example about this feature is that, on Linux, it is possible to turn on or turn off the keyboard capslock LED by writing to some /sys/class/leds file such as "$ echo 1 > /sys/class/leds/input7\:\:capslock/brightness" which turns on the Capslock LED. By writing 0 to it, the LED is turned off. This feature of Unix and Linux allows controlling the hardware with any programming language capable of reading and writing files, including Python and standard C++. I guess that they may be using standard PC hardware with industrial IO card. They may also use a single-board computer or custom board with x86 low-power variants processors built for embedded applications as SOC system-on-chip. One example of low power x86-SOC based processor is: https://www.cnx-software.com/2015/04/09/vortex86dx3-is-a-new-x86-soc-for-embedded-systems/ https://www.cnx-software.com/2015/04/09/vortex86dx3-is-a-new...

jchw 6y ago

Once you decide that absolute reliability is unattainable, you can instead aim for sufficient redundancy. It’s not perfect, but with enough safe guards in place it may very well be good enough.

microcolonel 6y ago
Software is not always like hardware though: the same software failure can cascade across redundant instances rapidly just as easily as it affects the primary.
- jchw 6y ago
  The same hardware failure can cascade too; redundancy alone is not a panacea, you need some safeguards. Ideally, nothing should fully trust anything else.
  - microcolonel 6y ago
    Maybe cascade was the wrong word. I meant that if you pop in a hot spare and it's running the same software, there's a good chance it will encounter the same issue, sometimes immediately.
    - perl4ever 6y ago
      In some cases redundant software is developed independently by different teams to the same spec; I think I read about that with the Space Shuttle. How correlated are your layers of swiss cheese...?
  - perl4ever 6y ago
    I was just reading that they design the rocket so that one engine could explode without affecting the others (hopefully) and they test the engines by dropping a nut or bolt or something in, which is not something rocket engines normally can tolerate.

ryder9 6y ago

you sure showed those people who worked hundreds of hours and have more knowledge of their system than some rando in this 'intellectual' website

7leafer 6y ago
you sure gave enough arguments to beat those of the parent.
- ryder9 6y ago
  not on me to prove his arguments stable genius

davedx 6y ago

Why not Javascript? I would argue that a good usable UI in a spacecraft is a good thing, and Javascript is a great language for building UIs

fdw 6y ago
> Javascript is a great language for building UIs I disagree. It is easy to build something presentable with HTML and browsers, and there have been many JS frameworks that lower the barrier. However, that is not because JS is a good language for it; it is because JS is the only language available in that environment. In fact, there are at least two other languages that transpile to JS, because you need JS to be able to run in a browser. But JS the language is not good enough, so you want two write your code in something else. Building a UI in a completely different language, without a browser, is definitely harder, but you might have several advantages in the other, non-UI parts.
- davedx 6y ago
  You haven’t made any arguments about why Javascript isn’t good for building UIs there. Here’s mine for why it is: - first class functions (callback style is a good fit for UI programming) - async/await and single threaded - many components of UIs are async by nature, Javascript supports you well here while keeping things simple with its single threaded model
  - vertex-four 6y ago
    There's... a large number of languages with both of those features.
    - scotty79 6y ago
      Could you name few better than js?
      - vertex-four 6y ago
        In the context of writing reliable software for a spacecraft? Pretty much anything with a reasonable type system, for a start. Maybe C#, or some reasonable subset of C++20, or Rust. Maybe Reason or OCaml.

raverbashing 6y ago

I don't see why. Chromium is being used in the UI. Not on critical flight software. > maybe even manual docking It isn't done manually > C++ and linux do not strike me as mission critical piece of software / language either A lot of flight software is done on C++, as you even mentioned. In the end, your best reliability partner is not using a language that's only know by fewer people. What would be a good "mission critical piece of software"? QNX (with no memory protection?). Linux is fine. I suppose they're not firing Ubuntu out of the box and running that with no customizations. Separation of concerns and redundancy usually beat correctness on the long term (which doesn't mean they didn't test the correctness of the system).

deleted 6y ago
[deleted]
StillBored 6y ago
QNX (with no memory protection?) Its probably a misunderstanding, but QNX has a very strong memory protection model that extends to its driver stack. Given that its a true realtime/microkernel, which has its own set of problems, but memory protection isn't one of them. For a number of cases QNX is going to be a large step up from linux. http://www.qnx.com/developers/docs/6.5.0/topic/com.qnx.doc.ide.userguide/topic/memory_MEMMGT_.html http://www.qnx.com/developers/docs/6.5.0/topic/com.qnx.doc.i...

mofosyne 6y ago

Unlikely that they are using javascript in the actual flight controller. Also under the screen they have a small amount of manual buttons that is likely to be seperate from the touchscreen interface logic.

nullifidian 6y ago
>javascript in the actual flight controller. zero chances that it's the case. I meant that in aerospace applications UI is mission critical too.
- eqvinox 6y ago
  > I meant that in aerospace applications UI is mission critical too. It is, but usability of the UI is mission critical as well. The humans in the capsule need to be able to quickly and effectively consume information in a way that doesn't make important facts hard to get at, and then quickly and effectively perform commands to address issues. I don't think a touch GUI is a bad choice in this regard, and if I'm building a touch GUI I'd rather use some existing framework. My personal choice would've been something like Gtk or Qt though :)

elteto 6y ago

In this case, C++ code is written in a different style than regular applications: exceptions are not allowed, allocations are not allowed past a certain point in program initialization, raw pointers are frowned upon, etc. This is common in this type of software and when you pair it with the right development culture it really makes a difference in the robustness of the result.

justinclift 6y ago
Any idea if there's a database layer (eg SQLite, PG, etc) on the systems?
- elteto 6y ago
  On the ground side, yes, there are many databases for different purposes. But not on the actual flight software.
  - exikyut 6y ago
    Very curious, how is state persisted where non-volatile storage is needed, and how is that persistence structured?
    - Nicksil 6y ago
      State is, as far as the system is concerned, is what values the multitude of sensors are reporting at any given moment.
    - elteto 6y ago
      It is never persisted outside of RAM. The flight computers are never powered off during a mission. You rely on two flight computers to always be functioning, that’s why the system is one-fault redundant. You can think as each string being “functional“: given the same set of inputs you expect the same set of outputs.
      - 6y ago

eqvinox 6y ago

I'm reasonably sure that quite a bit of the complexity is more or less directly caused by the amount of features it provides. I agree with you in that Chromium probably brings in a lot of features they don't need, but I would also say that reimplementing the features they do need is also a huge liability and would incur quite a bit of complexity too... but it'd make it their insular system with no other resources flowing into improving it. I don't have the data to make that call, but I would think someone at their end evaluated this quite thoroughly and determined the risks of rolling their own to exceed the risks of using Chromium. (Also they mentioned on the stream that they do in fact have a few hardware buttons for important things. Can't cite this, sorry...)

stefan_ 6y ago

The fact that it's implemented in JavaScript and Chromium just tells you the fancy display is entirely superfluous to mission success.

htk 6y ago
Interesting way to see it. For anyone wondering how unreliable it is, think instead that the trade off was well planned.
yellowapple 6y ago
Is there a backup should that fancy display fail? If not, then I'd hardly call it "superfluous to mission success".
- Czarcasm 6y ago
  Apparently the physical controls below the screens can perform all of the mission critical functions in the event of a display failure.
  - _emacsomancer_ 6y ago
    If only browsers had a similar fallback.

vijaybritto 6y ago

It should be most likely be WebGL and some glue js code. I don't think they're running everything in just js for the view code.

Nokinside 6y ago

Most safety critical code in the world, both civilian and military, is currently written in C/C++. Ada/SPARK is also used but it 's below 50%. These languages have the best practical static code analyzer, verification and proofing tools money can by. I'm personally using Astree https://www.absint.com/astree/index.htm https://www.absint.com/astree/index.htm Automatic docking software for the ATV that delivers supply to ISS is written using C code and verified with Astree. People get stuck into the language, but if it has some features that make it work, you can use it with verifiers. Then you write tests and simulators and tests ... JavaScript and Chromium seems more suspect to me, but if the code is well tested and has limited run time, I think it can be good to go. The state is stored and prosessed in those radiation hardened RAD750 processors. Chromium/JavaScript runtime image can restart every few seconds if needed. The UI and controls seem like unnecessarily flashy bullshit to me. Usually if you diverge from KISS principle bad things happen. But before I make any judgements I would see how they made those decisions.

thebruce87m 6y ago
How much is Astrée? I get nervous when you can’t see a price on the website.

edsac_xyzw 6y ago

In the case of SpaceX, it seems that they use embedded Linux and C++ with x86 processors, same as standard PC processors. A reason for using embedded linux is that, it allows using standard C++ or even scripting languages for controlling the hardware from the user-space by just reading and writing files. Linux device drivers (aka kernel modules) maps the hardware to special files on /sys or /dev. For instance, it makes possible to control a GPIO (General Purpose IO) which the device driver maps to sysfs special file system, by just writing 1 or 0 to the file a like /sys/class/gpio/gpio4/value which would enable GPIO 4 and turn on a LED attached to it. Another practical example about this feature is that, on Linux, it is possible to turn on or turn off the keyboard capslock LED by writing to some /sys/class/leds file such as "$ echo 1 > /sys/class/leds/input7\:\:capslock/brightness" which turns on the Capslock LED. By writing 0 to it, the LED is turned off. This feature of Unix and Linux allows controlling the hardware with any programming language capable of reading and writing files, including Python and standard C++. I guess that they may be using standard PC hardware with industrial IO card. They may also use a single-board computer or custom board with x86 low-power variants processors built for embedded applications as SOC system-on-chip. One example of low power x86-SOC based processor is: https://www.cnx-software.com/2015/04/09/vortex86dx3-is-a-new-x86-soc-for-embedded-systems/ https://www.cnx-software.com/2015/04/09/vortex86dx3-is-a-new...