6 ms·

The real point is at the end: > Finding general issues is a good thing, but other vendors are not subject to this level of scrutiny. We have no real (at least

The real point is at the end: > Finding general issues is a good thing, but other vendors are not subject to this level of scrutiny. We have no real (at least not this in depth) assurance that products from rival vendors are more secure." Pretty sure companies like Cisco (who have had recent high-profile security fails) would not want the same code review. Until they do, and someone can do a comparative analysis of the two sets of codebases, there are no useful conclusions to be taken by decision makers. The only value here is for firmware developers - learn from these mistakes. Hopefully GCHQ is coming for your code next.

raxxorrax 8y ago
> firmware developers - learn from these mistakes. Hopefully GCHQ is coming for your code next. I won't be able to sleep tonight, thank you very much... Lucky me I only develop for medical devices. Nobody seem to care about that too much... But firmware often does carry config-files for many, many different hardware revisions and versions. Creating a binary is mostly more work than just calling ./configure. This code is never nice code. It is also difficult to implement unit tests. So for efficient testing, you need a good laboratory. With many fancy hardware devices... Additionally, the code was probably meant to never get public. > these haven't been "universally applied" This is probably the case for any vendor that existed for more than half a year. To be honest, I can see the criticism, but there will be lacking "hygiene" at least at some cases. I wouldn't criticize the coders as much as the person responsible to use hardware from a chines vendor for critical infrastructure, that makes this kind of verification necessary. Why not build your own system then?
- calyth2018 8y ago
  > Nobody seem to care about that too much... Not until someone in a company has a great idea of hooking that up to the internet insecurely... Basically the medical equivalent of attaching the internet on Jeep's CAN bus...
  - raxxorrax 8y ago
    True. That creates sudden interest and with good reason. I don't have much experience with this but colleagues are furious. DICOM is often criticized and I know at least some manufacturers that have questionable security policies for online data exchange. I believe people are happy if it just works somehow, because some applied standards are older than TCP.
StudentStuff 8y ago
Cisco has gone through in depth code review for many of their products, as they are often the vendor of choice for security critical applications. Look at the desk phones in the upper echelons of the federal gov't and note how the "secure" systems are all Cisco with metal cages on the back to prevent signal leakage/sidechannel attacks. That being said, Cisco is a bag of dicks when it comes to their support contracts, pricing, and any of their consumer rubbish. Juniper and other 2nd tier hardware vendors are a bigger threat than Cisco IMO, their gear is much more common, and much more vulnerable (JunOS is a pile of security vulns). None of them take proper software development seriously, and the audits they've undergone are often just to rubber stamp their products. Frankly, none of these companies should be rolling their own "firmware", as they've proven that they won't reliably update as upstream (usually OpenWRT or Debian) push out security critical updates. Downgrading them to a role whereby their secret sauce is just an extra APT repo/extra metapackage from opkg is more than sufficient for what these companies need, while preventing them from accidentally or purposefully blocking updates.
- pleasecalllater 8y ago
  Yea, an in depth code review. Like the last Cisco router bug fixed by banning access if user-agent contains 'curl'?
  - raxxorrax 8y ago
    Wow, that is an amazing idea. That could keep out all those filthy hackers!
  - StudentStuff 8y ago
    I referenced this in my prior comment, calling out their consumer rubbish which they don't bother to maintain.
deleted 8y ago
[deleted]
DyslexicAtheist 8y ago
the amount of resources that went into QA at rival companies were pretty insane. Siemens COM, A/LU, Nokia, did levels of testing, people from pure software (no hw integration) backgrounds will find difficult to imagine ... copy-paste detection (which would have caught this), memory safety tests (valgrind/cachegrind/...), static code analysis (lint), unit-tests, code-coverage, peer-review, etc ... All this (testing) was done before we even started talking about "official testing" (e.g. system component test, blackbox test, whitebox test, integration test, ...) It was very much industry practice for NEVs since over a decade now. We made at least 1 member of every feature team sit through DFMEA training (pioneered by Motorola) and lived by these principles. I wish I'd had these huge resources (and freedom that comes with it to innovate) available in other projects. It's well known that Huawei competes on price not on quality so I guess pouring all that cash into QA doesn't make sense for them. (From my previous related comment[1] here) they are shit because they pitch to ICS and governments but are behaving like a start-up: > What hurt the established (big) guys most is that they actually thought like a start-up. If you worked at Huawei as a junior in Italy and other sites, you were allowed to move across departments and fields getting a well rounded picture of how the telco-sector works. If you wanted to gain the same experience with the big guys you'd have to stay 15-20 years in the company as opposed to 3-5 with Huawei. This is why Huawei engineers are highly sought after by other telco firms. [1] https://news.ycombinator.com/item?id=19025714 https://news.ycombinator.com/item?id=19025714

raxxorrax 8y ago

> firmware developers - learn from these mistakes. Hopefully GCHQ is coming for your code next. I won't be able to sleep tonight, thank you very much... Lucky me I only develop for medical devices. Nobody seem to care about that too much... But firmware often does carry config-files for many, many different hardware revisions and versions. Creating a binary is mostly more work than just calling ./configure. This code is never nice code. It is also difficult to implement unit tests. So for efficient testing, you need a good laboratory. With many fancy hardware devices... Additionally, the code was probably meant to never get public. > these haven't been "universally applied" This is probably the case for any vendor that existed for more than half a year. To be honest, I can see the criticism, but there will be lacking "hygiene" at least at some cases. I wouldn't criticize the coders as much as the person responsible to use hardware from a chines vendor for critical infrastructure, that makes this kind of verification necessary. Why not build your own system then?

calyth2018 8y ago
> Nobody seem to care about that too much... Not until someone in a company has a great idea of hooking that up to the internet insecurely... Basically the medical equivalent of attaching the internet on Jeep's CAN bus...
- raxxorrax 8y ago
  True. That creates sudden interest and with good reason. I don't have much experience with this but colleagues are furious. DICOM is often criticized and I know at least some manufacturers that have questionable security policies for online data exchange. I believe people are happy if it just works somehow, because some applied standards are older than TCP.

StudentStuff 8y ago

Cisco has gone through in depth code review for many of their products, as they are often the vendor of choice for security critical applications. Look at the desk phones in the upper echelons of the federal gov't and note how the "secure" systems are all Cisco with metal cages on the back to prevent signal leakage/sidechannel attacks. That being said, Cisco is a bag of dicks when it comes to their support contracts, pricing, and any of their consumer rubbish. Juniper and other 2nd tier hardware vendors are a bigger threat than Cisco IMO, their gear is much more common, and much more vulnerable (JunOS is a pile of security vulns). None of them take proper software development seriously, and the audits they've undergone are often just to rubber stamp their products. Frankly, none of these companies should be rolling their own "firmware", as they've proven that they won't reliably update as upstream (usually OpenWRT or Debian) push out security critical updates. Downgrading them to a role whereby their secret sauce is just an extra APT repo/extra metapackage from opkg is more than sufficient for what these companies need, while preventing them from accidentally or purposefully blocking updates.

pleasecalllater 8y ago
Yea, an in depth code review. Like the last Cisco router bug fixed by banning access if user-agent contains 'curl'?
- raxxorrax 8y ago
  Wow, that is an amazing idea. That could keep out all those filthy hackers!
- StudentStuff 8y ago
  I referenced this in my prior comment, calling out their consumer rubbish which they don't bother to maintain.

deleted 8y ago

[deleted]

DyslexicAtheist 8y ago

the amount of resources that went into QA at rival companies were pretty insane. Siemens COM, A/LU, Nokia, did levels of testing, people from pure software (no hw integration) backgrounds will find difficult to imagine ... copy-paste detection (which would have caught this), memory safety tests (valgrind/cachegrind/...), static code analysis (lint), unit-tests, code-coverage, peer-review, etc ... All this (testing) was done before we even started talking about "official testing" (e.g. system component test, blackbox test, whitebox test, integration test, ...) It was very much industry practice for NEVs since over a decade now. We made at least 1 member of every feature team sit through DFMEA training (pioneered by Motorola) and lived by these principles. I wish I'd had these huge resources (and freedom that comes with it to innovate) available in other projects. It's well known that Huawei competes on price not on quality so I guess pouring all that cash into QA doesn't make sense for them. (From my previous related comment[1] here) they are shit because they pitch to ICS and governments but are behaving like a start-up: > What hurt the established (big) guys most is that they actually thought like a start-up. If you worked at Huawei as a junior in Italy and other sites, you were allowed to move across departments and fields getting a well rounded picture of how the telco-sector works. If you wanted to gain the same experience with the big guys you'd have to stay 15-20 years in the company as opposed to 3-5 with Huawei. This is why Huawei engineers are highly sought after by other telco firms. [1] https://news.ycombinator.com/item?id=19025714 https://news.ycombinator.com/item?id=19025714