6 ms·

> Is the intellectual asymmetry between creating a design and analyzing it so large that a small teardown team can reliably analyze trade-offs in a design creat

> Is the intellectual asymmetry between creating a design and analyzing it so large that a small teardown team can reliably analyze trade-offs in a design created by a large engineering group? I don't understand this. So you are saying the reviewer's small team doesn't understand the expertise which might have gone in the car. But isn't that true for every product review and article out there? Take for example a phone. If a reviewer doubts the CPU benchmark on a phone, are we going to talk about how a single reviewer cannot understand the trade-off a bigger team at Apple/Samsung/Google has put? Or if a journalist talking about FB's decision on data sharing has the capacity to know the trade-offs made by a much larger technical and business group at FB. We can only look at the past work. If HN is voting this up then we know these guys have done some good work in the past. Unless of course we think everyone who doubts Tesla's expertise doesn't understand their genius.

jaggederest 9y ago
Munro and Associates is the firm that does this kind of analysis in the automotive industry. It's really fascinating. For the record, it would be hard to produce a design that Munro couldn't find something to talk about, good or bad. They employ extremely gifted engineers with a ridiculously large amount of experience at doing this exact task. As a software engineer, I wish there were an equivalent firm in the industry that did teardowns like this so that we could, as an industry, improve over time the way that the automotive industry does. I feel like it may be one of the factors holding software engineering back from achieving true professional engineering standardization.
- decacorn1 9y ago
  No they are not. Car companies do not go to Munro, period. There are far more reputable sources, speaking from someone in the industry. Munro is just trying to get free advertising with this sh*t.
  - luckydata 9y ago
    That's interesting. Which sources are in your view more reputable? Where do you work? One of the big three or a Japanese carmaker?
  - woodandsteel 9y ago
    Car companies do not go to Munro, period. Really? So where do they get their income? The comment below has some questions for you. This is all information you would have provided if you actually knew what you were talking about. The fact that you didn't leads me to think you are a troller who just likes to say things that make people angry.
  - justin66 9y ago
    > Car companies do not go to Munro, period. I don't know if you're trolling or what, but the truth is interesting. None of the US domestic automakers buy reports from Munro. They sell information to Japanese, Korean, Chinese and European automakers.
- digikata 9y ago
  I feel like you get a flavor of this for software by reading the mailing lists in the more mature open source project, you can basically get a continuous flow of analysis and build or rebuild. It's not packaged in a nice episodic report form, but you can get a little of that too with some projects. For the software industry as a whole, the closest thing might be the SEI[1], but that body of work takes some effort to work through, perhaps suffering a bit from a more adademic roots vs a more industry engineering info exchange approach like M&A. For that you might have specialist areas, for example, the Jepsen tests and blog[2]. [1] https://en.wikipedia.org/wiki/Software_Engineering_Institute https://en.wikipedia.org/wiki/Software_Engineering_Institute [2] https://aphyr.com/tags/Jepsen https://aphyr.com/tags/Jepsen
- alasdair_ 9y ago
  >As a software engineer, I wish there were an equivalent firm in the industry that did teardowns like this so that we could, as an industry, improve over time the way that the automotive industry does. There are quite a lot of security companies that reverse engineer other software products and publish their findings. It would be good to see that for non-security stuff too.
  - lithos 9y ago
    Do you want your bosses going to a firm like Monroe. Because companies will occasionally send in their own product to see if they can squeeze their manufacturers and see what their engineers missed.
    - jaggederest 9y ago
      Yes, honestly, I do. I would absolutely appreciate detailed and specific feedback across all of my work. How else are we expected to improve? Even when it's thing subject to opinion, I'd like to hear those other opinions.
- thisisit 9y ago
  > As a software engineer, I wish there were an equivalent firm in the industry that did teardowns like this so that we could, as an industry, improve over time the way that the automotive industry does. I think this thread is a good indicator of why that might never happen. People in software will argue with whatever stuff they can and try to refute a review/teardown - reviewer doesn't understand the design consideration, reviewer don't have numbers, the stuff is revolutionary etc etc.
Jaecen 9y ago
> If a reviewer doubts the CPU benchmark on a phone, are we going to talk about how a single reviewer cannot understand the trade-off a bigger team at Apple/Samsung/Google has put? The equivalent for the GP's claim would be the reviewer claiming that a phone's CPU is underpowered and that a more powerful one could have been used instead. That assumes a lot of knowledge about the tradeoffs that the phone designer had to consider - technical, logistical, financial, etc. I don't know anything about Monroe & Associates' expertise, but I agree with the GP that this is a high bar to clear.
megaremote 9y ago
> f a reviewer doubts the CPU benchmark on a phone, are we going to talk about how a single reviewer cannot understand the trade-off a bigger team at Apple/Samsung/Google has put? This makes no sense. Yes, if a reviewer doubts it, it means nothing unless they can prove it with numbers.
- deleted 9y ago
  [deleted]
fulafel 9y ago
It's the same ot worse for phones, yes.
tlb 8y ago
You don't need to be an expert to criticize the user experience of a product. A reviewer who says "this phone is slow" is probably right. It requires much deeper knowledge to criticize internal details. A reviewer who says "they used too big a capacitor for C143" would need to understand a great deal about the design for that claim to be credible. Most of the negatives in the article seemed to be of the second kind.

jaggederest 9y ago

Munro and Associates is the firm that does this kind of analysis in the automotive industry. It's really fascinating. For the record, it would be hard to produce a design that Munro couldn't find something to talk about, good or bad. They employ extremely gifted engineers with a ridiculously large amount of experience at doing this exact task. As a software engineer, I wish there were an equivalent firm in the industry that did teardowns like this so that we could, as an industry, improve over time the way that the automotive industry does. I feel like it may be one of the factors holding software engineering back from achieving true professional engineering standardization.

decacorn1 9y ago
No they are not. Car companies do not go to Munro, period. There are far more reputable sources, speaking from someone in the industry. Munro is just trying to get free advertising with this sh*t.
- luckydata 9y ago
  That's interesting. Which sources are in your view more reputable? Where do you work? One of the big three or a Japanese carmaker?
- woodandsteel 9y ago
  Car companies do not go to Munro, period. Really? So where do they get their income? The comment below has some questions for you. This is all information you would have provided if you actually knew what you were talking about. The fact that you didn't leads me to think you are a troller who just likes to say things that make people angry.
- justin66 9y ago
  > Car companies do not go to Munro, period. I don't know if you're trolling or what, but the truth is interesting. None of the US domestic automakers buy reports from Munro. They sell information to Japanese, Korean, Chinese and European automakers.
digikata 9y ago
I feel like you get a flavor of this for software by reading the mailing lists in the more mature open source project, you can basically get a continuous flow of analysis and build or rebuild. It's not packaged in a nice episodic report form, but you can get a little of that too with some projects. For the software industry as a whole, the closest thing might be the SEI[1], but that body of work takes some effort to work through, perhaps suffering a bit from a more adademic roots vs a more industry engineering info exchange approach like M&A. For that you might have specialist areas, for example, the Jepsen tests and blog[2]. [1] https://en.wikipedia.org/wiki/Software_Engineering_Institute https://en.wikipedia.org/wiki/Software_Engineering_Institute [2] https://aphyr.com/tags/Jepsen https://aphyr.com/tags/Jepsen
alasdair_ 9y ago
>As a software engineer, I wish there were an equivalent firm in the industry that did teardowns like this so that we could, as an industry, improve over time the way that the automotive industry does. There are quite a lot of security companies that reverse engineer other software products and publish their findings. It would be good to see that for non-security stuff too.
- lithos 9y ago
  Do you want your bosses going to a firm like Monroe. Because companies will occasionally send in their own product to see if they can squeeze their manufacturers and see what their engineers missed.
  - jaggederest 9y ago
    Yes, honestly, I do. I would absolutely appreciate detailed and specific feedback across all of my work. How else are we expected to improve? Even when it's thing subject to opinion, I'd like to hear those other opinions.
thisisit 9y ago
> As a software engineer, I wish there were an equivalent firm in the industry that did teardowns like this so that we could, as an industry, improve over time the way that the automotive industry does. I think this thread is a good indicator of why that might never happen. People in software will argue with whatever stuff they can and try to refute a review/teardown - reviewer doesn't understand the design consideration, reviewer don't have numbers, the stuff is revolutionary etc etc.

Jaecen 9y ago

> If a reviewer doubts the CPU benchmark on a phone, are we going to talk about how a single reviewer cannot understand the trade-off a bigger team at Apple/Samsung/Google has put? The equivalent for the GP's claim would be the reviewer claiming that a phone's CPU is underpowered and that a more powerful one could have been used instead. That assumes a lot of knowledge about the tradeoffs that the phone designer had to consider - technical, logistical, financial, etc. I don't know anything about Monroe & Associates' expertise, but I agree with the GP that this is a high bar to clear.

megaremote 9y ago

> f a reviewer doubts the CPU benchmark on a phone, are we going to talk about how a single reviewer cannot understand the trade-off a bigger team at Apple/Samsung/Google has put? This makes no sense. Yes, if a reviewer doubts it, it means nothing unless they can prove it with numbers.

deleted 9y ago
[deleted]

fulafel 9y ago

It's the same ot worse for phones, yes.

tlb 8y ago

You don't need to be an expert to criticize the user experience of a product. A reviewer who says "this phone is slow" is probably right. It requires much deeper knowledge to criticize internal details. A reviewer who says "they used too big a capacitor for C143" would need to understand a great deal about the design for that claim to be credible. Most of the negatives in the article seemed to be of the second kind.