4 ms·

Uh, tape out doesn't mean you've verified anything properly. Looking at the various Intel bugs found, I can't help sniggering a little. Verification mistakes h

Uh, tape out doesn't mean you've verified anything properly. Looking at the various Intel bugs found, I can't help sniggering a little. Verification mistakes happen all the time though. It's just a matter of when the bugs are found: if found later, it has much greater impact. On the 28nm issue, getting your design to synthesize to 28 nm depends on the architecture and your constraints. Trickier data paths make it harder to get it meeting timing. Where your IOs are placed, where your RAMs are placed and how much you have, each thing can make it harder and harder to synthesize. Adapteva doesn't have much RAM I believe.

_yosefk 11y ago
Well, yeah, I'm just saying it wasn't a picnic at 40nm or 90nm, either. It didn't go from a walk in the park to a nightmare at 28nm. Also I believe all the bugs in TFA are logical bugs where the circuit would have misbehaved in a logical simulation, not the kind where the circuit "flips zeros to ones" or vice versa because of physical implementation issues. In this sense advanced nodes make things worse only indirectly by enabling larger, more complex designs.
- gluggymug 11y ago
  Agreed. The bugs seem logical. It doesn't reflect well on the Intel verification effort if they were logical though. It means they didn't verify properly. One possible guess/excuse-for-stuffing-up is that tools don't always simulate correctly. There can be weird gotchas where simulation models and reality don't match.
nickpsecurity 11y ago
Are you joking? Intel uses (even invented) so much verification tech it's crazy. I'm talking more than any small vendor could hope to use as it would slow them down or take too much brains/training. Any bugs they're having are more likely due to size of their project or how custom the optimizations are. Here's some Intel techniques they use for verification: https://www7.in.tum.de/um/25/pdf/LimorFix-2.pdf https://www7.in.tum.de/um/25/pdf/LimorFix-2.pdf https://www.cl.cam.ac.uk/~jrh13/slides/nasa-14apr10/slides.pdf https://www.cl.cam.ac.uk/~jrh13/slides/nasa-14apr10/slides.p... IBM's papers on their verification system for POWER processors mention all kinds of optimizations for things like pipelines that make the logic ridiculous. Then they jump through hoops in verification. Yet, they don't hit 3+GHz on good pipeline without that. Undoubtedly, Intel's using similar tricks with similar issues. Regular ASIC verification doesn't cut it at their level. What they're doing is on another level. It's hard to say what exactly we should expect in terms of errata level given their operating constraints (esp marketing). Only thing I expect is to know clearly the circumstances errata appears so I can avoid it. They let me down...
- gluggymug 11y ago
  "Are you joking? Intel uses (even invented) so much verification tech it's crazy." That's kinda why I am sniggering. Intel has a rep for letting bugs go to silicon despite all that stuff. In terms of verification, they dropped the ball probably (unless they found these bugs in verification and decided to tape out anyway). One area where they fell on their face seems to be AVX. From TFA, "Certain Combinations of AVX Instructions May Cause Unpredictable System Behavior". That's a huge bug. Remember how we discussed the torture testing of floating point on the RISC-V ? Kinda similar issue. A customer wouldn't be happy with a huge bug there. The big 3 of EDA tools (Cadence, Synopsis, Mentor Graphics) provide their own tools for verifying this stuff called UVM. Like anything, it still relies on the person using the tool. It takes a lot of effort and planning to use this stuff. Whenever the verification engineer has to create Verification IP to test the IP, there's a chance they create bugs of their own. It's like a golden rule. That's why I am not a fan of formal methods. Nothing is proven until you have it working in silicon.
  - nickpsecurity 11y ago
    "That's why I am not a fan of formal methods. Nothing is proven until you have it working in silicon." The one's that did use formal methods all the way did what they were supposed to and usually first pass. They were a mix of academic and defense-related stuff most people can't buy. What I normally see when I look up formal verification in industry is equivalence checking with custom shops also doing protocol verification and certain correctness angles. We've seen lots of what Intel does in their docs. So, that narrows the question down to "Why are these errata in there anyway?" "The big 3 of EDA tools (Cadence, Synopsis, Mentor Graphics) provide their own tools for verifying this stuff called UVM. Like anything, it still relies on the person using the tool. It takes a lot of effort and planning to use this stuff." Didn't know about that one. Thanks. The briefs I just Googled sound weaker than Intel's stuff and especially IBM's where presentations cover an incredible amount of specific verifications. Like you said, what one puts in determines what one gets out of it. So, are Intel just being lax on verification or is their stuff just too complex + optimized to catch all the corner cases? "Intel has a rep for letting bugs go to silicon despite all that stuff. " If it's intentional and avoidable, then I think it might be wise in another light. (Or not, but worth considering.) The other light is the Lipner essay on why shipping is more important than highest quality: https://blogs.microsoft.com/cybertrust/2007/08/23/the-ethics-of-perfection/ https://blogs.microsoft.com/cybertrust/2007/08/23/the-ethics... That comes from a background where he and Karger did high assurance systems that aimed for perfection and got as close as they could in that period. Kept slipping behind competition in terms of features/speed/price and that would effect market share. So, his prior employer canceled that product with his next one following his recommendation to hit acceptable quality levels, ship, and continuosly improve the product. Wonder if Intel is doing that to keep market dominance? "Nothing is proven until you have it working in silicon." This we agree on. The formally verified stuff usually works first try but that's thanks to billions in R&D in tooling & fabs they used. I like knowing a batch of chips performed exactly according to spec when probed during operation. Funny I can't remember what you HW people call that activity. Anyway, I'd love to email or chat with you sometime to see an insider's view on this topic and fill in some blanks. Reason being experienced ASIC people talk very little vs software people. I'm collecting what tidbits of reality I can for a variety of reasons. Two important ones are giving a head start to people aiming for HW design and boosting high assurance design by determining where the weak points are currently. Really busy right now but maybe later on, eh?

_yosefk 11y ago

Well, yeah, I'm just saying it wasn't a picnic at 40nm or 90nm, either. It didn't go from a walk in the park to a nightmare at 28nm. Also I believe all the bugs in TFA are logical bugs where the circuit would have misbehaved in a logical simulation, not the kind where the circuit "flips zeros to ones" or vice versa because of physical implementation issues. In this sense advanced nodes make things worse only indirectly by enabling larger, more complex designs.

gluggymug 11y ago
Agreed. The bugs seem logical. It doesn't reflect well on the Intel verification effort if they were logical though. It means they didn't verify properly. One possible guess/excuse-for-stuffing-up is that tools don't always simulate correctly. There can be weird gotchas where simulation models and reality don't match.

nickpsecurity 11y ago

Are you joking? Intel uses (even invented) so much verification tech it's crazy. I'm talking more than any small vendor could hope to use as it would slow them down or take too much brains/training. Any bugs they're having are more likely due to size of their project or how custom the optimizations are. Here's some Intel techniques they use for verification: https://www7.in.tum.de/um/25/pdf/LimorFix-2.pdf https://www7.in.tum.de/um/25/pdf/LimorFix-2.pdf https://www.cl.cam.ac.uk/~jrh13/slides/nasa-14apr10/slides.pdf https://www.cl.cam.ac.uk/~jrh13/slides/nasa-14apr10/slides.p... IBM's papers on their verification system for POWER processors mention all kinds of optimizations for things like pipelines that make the logic ridiculous. Then they jump through hoops in verification. Yet, they don't hit 3+GHz on good pipeline without that. Undoubtedly, Intel's using similar tricks with similar issues. Regular ASIC verification doesn't cut it at their level. What they're doing is on another level. It's hard to say what exactly we should expect in terms of errata level given their operating constraints (esp marketing). Only thing I expect is to know clearly the circumstances errata appears so I can avoid it. They let me down...

gluggymug 11y ago
"Are you joking? Intel uses (even invented) so much verification tech it's crazy." That's kinda why I am sniggering. Intel has a rep for letting bugs go to silicon despite all that stuff. In terms of verification, they dropped the ball probably (unless they found these bugs in verification and decided to tape out anyway). One area where they fell on their face seems to be AVX. From TFA, "Certain Combinations of AVX Instructions May Cause Unpredictable System Behavior". That's a huge bug. Remember how we discussed the torture testing of floating point on the RISC-V ? Kinda similar issue. A customer wouldn't be happy with a huge bug there. The big 3 of EDA tools (Cadence, Synopsis, Mentor Graphics) provide their own tools for verifying this stuff called UVM. Like anything, it still relies on the person using the tool. It takes a lot of effort and planning to use this stuff. Whenever the verification engineer has to create Verification IP to test the IP, there's a chance they create bugs of their own. It's like a golden rule. That's why I am not a fan of formal methods. Nothing is proven until you have it working in silicon.
- nickpsecurity 11y ago
  "That's why I am not a fan of formal methods. Nothing is proven until you have it working in silicon." The one's that did use formal methods all the way did what they were supposed to and usually first pass. They were a mix of academic and defense-related stuff most people can't buy. What I normally see when I look up formal verification in industry is equivalence checking with custom shops also doing protocol verification and certain correctness angles. We've seen lots of what Intel does in their docs. So, that narrows the question down to "Why are these errata in there anyway?" "The big 3 of EDA tools (Cadence, Synopsis, Mentor Graphics) provide their own tools for verifying this stuff called UVM. Like anything, it still relies on the person using the tool. It takes a lot of effort and planning to use this stuff." Didn't know about that one. Thanks. The briefs I just Googled sound weaker than Intel's stuff and especially IBM's where presentations cover an incredible amount of specific verifications. Like you said, what one puts in determines what one gets out of it. So, are Intel just being lax on verification or is their stuff just too complex + optimized to catch all the corner cases? "Intel has a rep for letting bugs go to silicon despite all that stuff. " If it's intentional and avoidable, then I think it might be wise in another light. (Or not, but worth considering.) The other light is the Lipner essay on why shipping is more important than highest quality: https://blogs.microsoft.com/cybertrust/2007/08/23/the-ethics-of-perfection/ https://blogs.microsoft.com/cybertrust/2007/08/23/the-ethics... That comes from a background where he and Karger did high assurance systems that aimed for perfection and got as close as they could in that period. Kept slipping behind competition in terms of features/speed/price and that would effect market share. So, his prior employer canceled that product with his next one following his recommendation to hit acceptable quality levels, ship, and continuosly improve the product. Wonder if Intel is doing that to keep market dominance? "Nothing is proven until you have it working in silicon." This we agree on. The formally verified stuff usually works first try but that's thanks to billions in R&D in tooling & fabs they used. I like knowing a batch of chips performed exactly according to spec when probed during operation. Funny I can't remember what you HW people call that activity. Anyway, I'd love to email or chat with you sometime to see an insider's view on this topic and fill in some blanks. Reason being experienced ASIC people talk very little vs software people. I'm collecting what tidbits of reality I can for a variety of reasons. Two important ones are giving a head start to people aiming for HW design and boosting high assurance design by determining where the weak points are currently. Really busy right now but maybe later on, eh?