22 ms·

I have worked in card payment industry. We would be getting products from China with added boards to beam credit card information. This wasn't state-sponsored a

I have worked in card payment industry. We would be getting products from China with added boards to beam credit card information. This wasn't state-sponsored attack. Devices were modified while on production line (most likely by bribed employees) as once they were closed they would have anti-tampering mechanism activated so that later it would not be possible to open the device without setting the tamper flag. Once this was noticed we started weighing the terminals because we could not open the devices (once opened they become useless). They have learned of this so they started scraping non-essential plastic from inside the device to offset the weight of the added board. We have ended up measuring angular momentum on a special fixture. There are very expensive laboratory tables to measure angular momentum. I have created a fixture where the device could be placed in two separate positions. The theory is that if the weight and all possible angular momentums match then the devices have to be identical. We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker.

RL_Quine 8y ago
Get an x-ray machine? They are surprisingly cheap pieces of hardware if you are willing to deal with a small area, low penetration image. Low penetration means no lead, which makes for something that's about as cumbersome as a large bar fridge.
- tuanva 8y ago
  It's mentioned in the article that X-Ray didn't help much: 'Gray or off-white in color, they looked more like signal conditioning couplers, another common motherboard component, than microchips, and so they were unlikely to be detectable without specialized equipment'
  - RL_Quine 8y ago
    This comment is specific to the parent talking about their experiences producing credit card terminals that ended up with PCBs implanted in them. Here it is appropriate.
    - ksec 8y ago
      And as soon as they know you are using X-Ray to test ( I would be surprised if there were no insider information leaking on the OP's end ), they would develop a way so that somehow the X-Ray images looks identical. May be a heat spreader like metal as cover up.
  - PascLeRasc 8y ago
    Can't the image be diffed with one of a trusted system?
jypepin 8y ago
First, wow this is both incredible and crazy! Both the China-side hacks and your side's anti-hack. Mind. Blown. Second, would have it been cheaper to manufacture somewhere more trustworthy (another country?) instead of spending all this time/money on your anti-hack systems?
- m_mueller 8y ago
  > Second, would have it been cheaper to manufacture somewhere more trustworthy (another country?) instead of spending all this time/money on your anti-hack systems? I'd like to know this too. Has the West completely lost the ability to mass produce microchips at even a reasonable cost for financial applications?
  - TazeTSchnitzel 8y ago
    It's not the chips that are the problem. Most of Intel's fabs are in the US, and their assembly sites are in a number of countries.
    - cwingrav 8y ago
      I think people should know how stark the differences for assembly in the US are vs outside of it. Something that costs, at low prototype volumes mind you, $20 in China for a dozen boards or so, would run hundreds of dollars in the US and still take the same amount of time. As it scales up, the ratio might improve, but these aren't like 10%-20% differences.
  - raverbashing 8y ago
    This is not microchips, this is basic PCB assembly But I'd guess momentum is hard to change.
  - baybal2 8y ago
    America has fabs, both old and leading edge, but ask industry giants like Gemalto to even bother to manufacture chips anywhere outside of Taiwan, assemble the final product outside of China. They will never do that, because they look for the cheapest solution. The bigger the company, the less it cares about things other than cost. This is why Mediatek and Broadcom can usurp the market of network SoCs, while making products with atrociously bad support. I personally dealt with both, and say that they wholely match their popular culture image. I don't know how it is with USA, but for Russia, the military doesn't care that their chips had frequency measured in kilohertz, and had sizes measured in square sentimetres, for as long as they get them made inside the country.
denyslb 8y ago
Sorry, but this doesn't sound as true or there is huge mistakes done in choosing supply chain for such sensitive matter. How come company keep ordering devices from some unverified sources from China, and after hitting a wall keep doing same? How do you accept shipment of such devices without randomly opening and inspecting sample(yes losing all data, but electronic inspection can be done). How you didn't investigate that with Visa/Mastercard? Whoever does that, he will lose his payment terminal certification after such incident, because they will track them down by IC serials very quickly. What if vendor changed power supply board or even components type on it, and your momentum or weight test will make false positive? Unless... your employer or you buy single devices, on demand, from some shady aliexpress seller. But then, it is plain suicide.
- gm-conspiracy 8y ago
  It sounds like they are just ordering stock products from Amazon.
- lmilcin 8y ago
  While it may sound sensational this was more of an operational issue, really. We were told by Visa and Mastercard that it is not even a question if we are going to be targeted. If you work in payment card industry you are constantly being attacked and the only way is to make it part of the process to deal with those things. Our network was hacked but what was important was tight, almost mathematical processes around protecting very specific material like credit card data and PINs. For example, PINs are only ever being in unencrypted form inside of Hardware Security Modules and only for the purpose of being encrypted with Visa/Mastercard exchange keys. The process was designed so that nobody has enough access to ever get enough cryptographic material to be able to decrypt anything, at least two or three people would have to collude to do anything. It also happens that we put all our resources in investment in software for the platform locking ourselves in. It would be rash decision to change the platform and it would probably kill our company. Also we (correctly) gambled that it would be dealt with quickly.
  - deleted 8y ago
    [deleted]
rqs 8y ago
Why don't you guys consider to expose this by suing the bad manufacturer? I believe this could help other truely honest manufacturers both in and outside China to beat the wrong doers.
- patrickg_zill 8y ago
  The Chinese operation would simply shut down and reopen under a different name. And the credit card companies are always very worried about their brand image, so they are not interested in any negative publicity.
  - rqs 8y ago
    But I don't think it's that good to keep playing the rat-cat game. > the credit card companies are always very worried about their brand image, so they are not interested in any negative publicity. Letting the bad guys keep doing what they do while putting it's own customers under shadow, to me, it sounds like the credit card companies are helping the bad guys. As a Chinese myself and a credit card user, I'm more worried than you guys do, because we are more likely to receive that kind of treatment (compromised credit card and computer chips etc). I really hope somebody can teach those bad doers a unforgettable lesson.
TheSpiceIsLife 8y ago
This is fairly analogous to my comment here: https://news.ycombinator.com/item?id=18138847 https://news.ycombinator.com/item?id=18138847 Presumably the organisations responsible for hard coding backdoors in chip designs know how to test to confirm their presence. Presumably some adversarial nation-states have moles inside these organisations > know how to remove them prior to fab. Presumably these adversaries export genuine chips to their adversaries, thereby tricking them in to thinking the backdoors made it through the fab process, and only use chips that have the backdoors removed in their own critical infrastructure. Presumably. I’ve always had this fantasy of being a hextuple agent involved in this type of deep espionage.
jsjohnst 8y ago
> as once they were closed they would have anti-tampering mechanism activated so that later it would not be possible to open the device without setting the tamper flag You didn’t specify what type of anti-tamper was used, but I wanted to jump in and say usually that means nothing. The US government intercepted packages [0] and put in back doors (removing and replacing the seals), so I’m not sure why you were so quick to dismiss state sponsored attacks by something as simple as an anti-tamper seal. You can learn how to do it yourself at most medium to large hacker conferences too (DEFCON, BlackHat, HOPE, and CCC to name a few, but there’s more with it). [0] https://www.techradar.com/news/networking/routers-storage/photos-reveal-nsa-tampered-with-cisco-router-prior-to-export-1249191 https://www.techradar.com/news/networking/routers-storage/ph...
- folli 8y ago
  The chinese government is probably not interested enough in credit card numbers to warrant involvement.
  - lmilcin 8y ago
    It would be foolish to show your hand for few pennies.
    - jandrese 8y ago
      Might make a difference if those pennies can't be tracked back to the government.
      - malandrew 8y ago
        Yup. This would be the modern equivalent of Air America and other schemes by the CIA to raise money to operate by involving themselves in illegal activity like the drug trade.
        jessaustin 8y ago
        One would consider those activities to have taken place in "modern" times as well...
pvarangot 8y ago
Maybe a setup to measure inertia tensor and center of mass (in that setup's axes) will be easier and I think it's what your call "measuring all angular momentums".
agumonkey 8y ago
Gosh, I would love to read everything about this. I know there are some videos about anti-tampering card readers on youtube, but not on the feedback race between hackers and security.
usrusr 8y ago
A truly sophisticated attacker would leave some easily detectable bad eggs in the stream after countering a countermeasure like the weighing.
ycmbntrthrwaway 8y ago
> We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker. You only need to measure three angular momentums, all other can be calculated. See https://en.wikipedia.org/wiki/Moment_of_inertia#Motion_in_space_of_a_rigid_body,_and_the_inertia_matrix https://en.wikipedia.org/wiki/Moment_of_inertia#Motion_in_sp... "This shows that the inertia matrix can be used to calculate the moment of inertia of a body around any specified rotation axis in the body." On the attacker side, they only need to make sure three angular momentums match.
- gnode 8y ago
  Maybe they're also measuring centrifugal force for rotations with axes not intersecting the centre of mass.
ramshanker 8y ago
I believe you are talking about "Rotational Moment of Inertia" about various axis, instead of "Angular Momentum". Angular Momentum of a body at rest is zero.
mycall 8y ago
Did you look into Terahertz scanning?
aarongough 8y ago
Very interesting methods you used to detect the changes! Out of curiosity: was there a reason that taking an xray of the devices was not an option? Industrial/veterinary xray machines can often be had quite cheaply...
amluto 8y ago
> We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker. You mean moment of inertia, not angular momentum. You could measure all of them! Given the moments for the three principal axes at any point, you can use the parallel axis theorem to calculate all the rest. In general, there are 10 degrees of freedom: 3 for the position of the center of mass, 3 for the axes, three for the moments, and for the the total mass. For a nicer way to count and to do the math, you have the inertia tensor at the CM (a 3-dimensional rank-2 symmetric tensor, 6 DoF) plus the location of the CM and the mass. In any event, this is a cute tampering-detection trick, but I would have started with an X-ray or CT scan.
- whatshisface 8y ago
  Another implication of the parallel axis theorem is that the attacker could perfectly mimic every moment of inertia by shaving plastic. They wouldn't have to know which two axes were being tested because there are only three real numbers worth of information in the system to begin with (once center of mass and total mass have been dealt with.) In the whole MOI tensor there are only six free numbers which sounds like a reasonable number of parameters to fix by adding and removing small amounts of material. What was this company doing in hiring an untrustworthy manufacturer to handle secure devices? That's playing a game you've lost from the start. Not every problem is technical!
  - lmilcin 8y ago
    I honestly did not know about that. I thought that if you move any mass (remove non zero mass and place it somewhere else) there must be at least one axis which you can use to detect the change in moment of inertia.
    - ISL 8y ago
      whatshisface is correct.
    - whatshisface 8y ago
      If the mass had to all move to one other place this would be correct (the center of mass would have to change), but the attackers are able to move the mass to multiple other places.
ryandrake 8y ago
Could this be solved by a clause in the contract that specified if any randomly sampled devices came tampered with, the manufacturer did not get paid?
- lmilcin 8y ago
  Ah, yes, and then we would voluntarily go out of business. And the supplier will just have another customer that would not be so principled.
  - ryandrake 8y ago
    Huh? This would be between you and the Chinese manufacturer. If you found s tampered unit, you don’t pay them. How should that make you go out of business?
    - pjc50 8y ago
      They need the units from somewhere, with a deadline determined by their own runway, and presumably only some of the Chinese ones were compromised. Sure, they could avoid paying, but then they have to spend months getting the production run going somewhere else .. which might also be compromised.
- fma 8y ago
  I'm sure the Chinese government would be willing to keep afloat companies that allow them to tamper with.
village-idiot 8y ago
At what point does it get cheaper to buy from sources that are more expensive but more trustworthy?
sonnyblarney 8y ago
Maybe time to buy your financial hardware from somewhere not china?
- cyc115 8y ago
  I know it's nice to blame China for everything... but it's not really the root of the problem here, supply chain management and control is.
  - sonnyblarney 8y ago
    China is 100% to blame here.
    - objektif 8y ago
      Absolutely.
Gravityloss 8y ago
Why do you use those suppliers?
wnevets 8y ago
That's kinda amazing
MaxBarraclough 8y ago
So you had hard evidence that your manufacturer was vandalising your property? Couldn't you drop them?
- hinkley 8y ago
  The main lesson I learned from my Dad’s employer (twice) was always have two vendors. You can play them off of each other. When I did contract work I saw how powerful getting out or putting for away the checkbook can be. Large vendors ignore you if the checkbook isn’t moving. I say “twice” because they were also the biggest employer in town. I got out of there. Slim pickins for career opportunities that didn’t revolve around BigCo.
onepremise 8y ago
Seriously, why are we still outsourcing chip manufacturing to other countries? Sure it's cheaper, but we sacrifice a lot to have a society of corporate slaves build our tech. Security, core domain knowledge, capability, corporate secrets, patent rewards and enforcement, etc... All of it you throw away the minute you ship your manufacturing out of the country. I've seen enough board printing machines out there to start working on our own. As a country, we need to close this gap, more automation and capability and there will be no need to outsource circuit board printing and manufacturing. We will be much better off.
- adamrezich 8y ago
  It's hard to compete domestically against low prices caused by China's completely different standards for wages and human rights for laborers. Plus, here we don't like the idea of manufacturing industries being subsidized by the government (except in the case of "defense" of course...), while China obviously has no qualms with doing so.
  - LargeWu 8y ago
    > except in the case of "defense" of course... Well, this thread is exactly the reason we subsidize the defense industry to make sure things are produced in the US.
- TheRealPomax 8y ago
  Counterpoint: even if we ignored the fact that you cannot possibly produce the volumes of chips necessary at the price necessary in your country rather than in "we don't have to acknowledge all the human rights violations" countries, why would you believe this problem goes away if chip manufacturing were done in your own country, rather than another? The moment the option of taking control of a production line of something _this important_ becomes available, your local specialized organized crime outfits will start to figure out ways to insert themselves into those production lines, learning the ins and outs, and figuring out a way to get something, anything, in there that won't be noticed but will give them a hook into millions of systems. The law does not prevent crime. It just puts a price on it. While that price is typically too high for individuals, for organizations that have no business registration to revoke, and no CEO to drag to court, it is an entirely trivial cost.
jiveturkey 8y ago
x-ray would have been easier, as others have said. but you're talking about the addition of an entire board! probably on the order of 10% of the size of the main boards. in this article, perhaps dumbed down or altered, they are talking about the addition of a single, tiny chip, too small to even be an MCU let alone have wireless capability (which BTW requires an antenna).
localhost 8y ago
For the folks commenting below that we should bring the manufacturing back to the US, why wouldn't the bad guys just start bribing American workers to insert the attack hardware into devices made here? It's not like Americans are somehow above being bribed.
- assblaster 8y ago
  Americans can be bribed to do anything. However, you don't really need to bribe or ask someone to betray their country and fellow citizens when it's done on Chinese soil...
- lev99 8y ago
  The bad actors could be brought to trial in a United States court, which is a level of deterrent not included in offshore manufacturing. If a US manufacturer was found selling tampered chips then the company itself could be held liable. This would create a general pressure to increase onsight security. There are other benefits to us based manufacturing, but you only brought up the crime aspect so I will leave the other benefits unsaid.
- s73v3r_ 8y ago
  It's possible, but it's much, much, much easier for an American company to hold another American company accountable when something like this happens. Instead of having to go through all those hoops, you sue the pants off the manufacturer.
- pfschell 8y ago
  Ignoring the abject amorality and greed that underpins Chinese culture will leave one confused like this.
  - roguecoder 8y ago
    I think people here are just suggesting that the folks who believe producing things in America would avoid a culture of abject amorality and greed seem naive, at best. Even ignoring the current obvious examples, Americans frequently engage in profitable criminal behavior: https://www.businessinsider.com/mcdonalds-monopoly-game-rigged-scam-report-2018-7 https://www.businessinsider.com/mcdonalds-monopoly-game-rigg...
deleted 8y ago
[deleted]
zelon88 8y ago
Wait a minute... So your company has a Chinese equipment supplier, finds out that the supplier is tampering with your purchased equipment, and your solution is to add criteria to the incoming inspection? No wonder China keeps screwing with you guys. You aren't supposed to eat that cost! Write a PO with tons of fine print that says "We will disassembly units at random for compliance inspection. Non compliant products will be returned at the suppliers expense." And then add a clause that says ">3 non-compliance events in under # months will result in the entire PO (10 or 20 units) being returned and all contracts cancelled." I cannot believe you are getting screwed by a company you choose to do business with and yet you eat cost to ensure they aren't screwing you. Just get a new supplier! Do on-site inspections at their facility. This is nuts.
- lmilcin 8y ago
  There were other considerations like the fact we were actually buing it from large reputable company and what happened was that some employees were doing it with no involvement of the company. The fact is, doing any kind of hardware production in China, you have to be aware Chineese have different value system and you would not be suited doing any business if you throw tantrum at any sign of apparent dishonesty (assuming the company was involved which they could not have been as they have been the ones damaged the most). If the company does screw you (like replacing components for something cheaper) they typically will not be thinking they are doing anything wrong. They are just testing if you notice and if you do not they will say it makes no difference for you but saves them costs. The way to work is then verify everything and politely point it out. If you notice they will correct apparent mistake.
  - SilasX 8y ago
    >There were other considerations like the fact we were actually buying it from large reputable company and what happened was that some employees were doing it with no involvement of the company. It's their responsibility to police for that. From the perspective of their business arrangement with you, it doesn't matter whether their left hand or right hand is evil; it's not your problem either way. Is it really hard to find alternatives? Are they really cheaper than non-Chinese sources when you account for these inspection costs?
kumarvvr 8y ago
This reminds me of a story about, IIRC, Soviet intelligence personnel determining that a photocopy machine at their consulate in the USA was bugged by the CIA by measuring it's weight and comparing that value to the standard value published by Xerox. Modern times, but same old methods of "debugging" The angular momentum stuff is innovative though.
- hinkley 8y ago
  Now I’m picturing a 300lbs photocopier being spun on three axes. How much tape would it take to keep doors and parts from flying off?
- 8note 8y ago
  is measuring the moment of inertia not standard? I think any mechanical engineer would suggest it for that kind of problem (disclaimer; I've suggested it before for the same problem)
clubm8 8y ago
>measuring angular momentum on a special fixture Can you elaborate on this concept a bit? I'm not familiar with this term and the sources I looked up were pretty physics-y and out of my depth. This sounds like a very interesting and creative solution. Good lateral thinking on your part! :)
mmaunder 8y ago
This feels cargo cultish. Products drop from the sky. One day they become poisonous. You have no idea how to reproduce them locally. So you come up with hacks to make then less dangerous. We really need to get back into manufacturing if this is our brave new world.
- ticviking 8y ago
  The better question is how did we stop manufacturing.
amelius 8y ago
What good is angular momentum when the producer can have fluctuations in its supply chain? Yes you can see when devices are not the same, but what if that happens all the time, legitimately?
badwabbut 8y ago
I do not for one minute believe, that china, whose production floors are spotless, watched like a hawk and every product tested, did not know this was being added. They would have been able to tell themselves by weight. Yes, product weight is used for shipping manifests too. This is obfuscation of the fact, that china as a state actor has perpetrated this crime against our country period. full stop.
greglindahl 8y ago
Thanks for sharing this story, and I hope you aren't put off by the huge thread of people second-guessing your competence. We need more of your kind of story.
ndesaulniers 8y ago
Was the secret service contacted? Card skimmers are a big no no. Family friend works for Dept of weights and measures and finds skimmers all the time on gas pumps. Scary stuff.

RL_Quine 8y ago

Get an x-ray machine? They are surprisingly cheap pieces of hardware if you are willing to deal with a small area, low penetration image. Low penetration means no lead, which makes for something that's about as cumbersome as a large bar fridge.

tuanva 8y ago
It's mentioned in the article that X-Ray didn't help much: 'Gray or off-white in color, they looked more like signal conditioning couplers, another common motherboard component, than microchips, and so they were unlikely to be detectable without specialized equipment'
- RL_Quine 8y ago
  This comment is specific to the parent talking about their experiences producing credit card terminals that ended up with PCBs implanted in them. Here it is appropriate.
  - ksec 8y ago
    And as soon as they know you are using X-Ray to test ( I would be surprised if there were no insider information leaking on the OP's end ), they would develop a way so that somehow the X-Ray images looks identical. May be a heat spreader like metal as cover up.
- PascLeRasc 8y ago
  Can't the image be diffed with one of a trusted system?

jypepin 8y ago

First, wow this is both incredible and crazy! Both the China-side hacks and your side's anti-hack. Mind. Blown. Second, would have it been cheaper to manufacture somewhere more trustworthy (another country?) instead of spending all this time/money on your anti-hack systems?

m_mueller 8y ago
> Second, would have it been cheaper to manufacture somewhere more trustworthy (another country?) instead of spending all this time/money on your anti-hack systems? I'd like to know this too. Has the West completely lost the ability to mass produce microchips at even a reasonable cost for financial applications?
- TazeTSchnitzel 8y ago
  It's not the chips that are the problem. Most of Intel's fabs are in the US, and their assembly sites are in a number of countries.
  - cwingrav 8y ago
    I think people should know how stark the differences for assembly in the US are vs outside of it. Something that costs, at low prototype volumes mind you, $20 in China for a dozen boards or so, would run hundreds of dollars in the US and still take the same amount of time. As it scales up, the ratio might improve, but these aren't like 10%-20% differences.
- raverbashing 8y ago
  This is not microchips, this is basic PCB assembly But I'd guess momentum is hard to change.
- baybal2 8y ago
  America has fabs, both old and leading edge, but ask industry giants like Gemalto to even bother to manufacture chips anywhere outside of Taiwan, assemble the final product outside of China. They will never do that, because they look for the cheapest solution. The bigger the company, the less it cares about things other than cost. This is why Mediatek and Broadcom can usurp the market of network SoCs, while making products with atrociously bad support. I personally dealt with both, and say that they wholely match their popular culture image. I don't know how it is with USA, but for Russia, the military doesn't care that their chips had frequency measured in kilohertz, and had sizes measured in square sentimetres, for as long as they get them made inside the country.

denyslb 8y ago

Sorry, but this doesn't sound as true or there is huge mistakes done in choosing supply chain for such sensitive matter. How come company keep ordering devices from some unverified sources from China, and after hitting a wall keep doing same? How do you accept shipment of such devices without randomly opening and inspecting sample(yes losing all data, but electronic inspection can be done). How you didn't investigate that with Visa/Mastercard? Whoever does that, he will lose his payment terminal certification after such incident, because they will track them down by IC serials very quickly. What if vendor changed power supply board or even components type on it, and your momentum or weight test will make false positive? Unless... your employer or you buy single devices, on demand, from some shady aliexpress seller. But then, it is plain suicide.

gm-conspiracy 8y ago
It sounds like they are just ordering stock products from Amazon.
lmilcin 8y ago
While it may sound sensational this was more of an operational issue, really. We were told by Visa and Mastercard that it is not even a question if we are going to be targeted. If you work in payment card industry you are constantly being attacked and the only way is to make it part of the process to deal with those things. Our network was hacked but what was important was tight, almost mathematical processes around protecting very specific material like credit card data and PINs. For example, PINs are only ever being in unencrypted form inside of Hardware Security Modules and only for the purpose of being encrypted with Visa/Mastercard exchange keys. The process was designed so that nobody has enough access to ever get enough cryptographic material to be able to decrypt anything, at least two or three people would have to collude to do anything. It also happens that we put all our resources in investment in software for the platform locking ourselves in. It would be rash decision to change the platform and it would probably kill our company. Also we (correctly) gambled that it would be dealt with quickly.
- deleted 8y ago
  [deleted]

rqs 8y ago

Why don't you guys consider to expose this by suing the bad manufacturer? I believe this could help other truely honest manufacturers both in and outside China to beat the wrong doers.

patrickg_zill 8y ago
The Chinese operation would simply shut down and reopen under a different name. And the credit card companies are always very worried about their brand image, so they are not interested in any negative publicity.
- rqs 8y ago
  But I don't think it's that good to keep playing the rat-cat game. > the credit card companies are always very worried about their brand image, so they are not interested in any negative publicity. Letting the bad guys keep doing what they do while putting it's own customers under shadow, to me, it sounds like the credit card companies are helping the bad guys. As a Chinese myself and a credit card user, I'm more worried than you guys do, because we are more likely to receive that kind of treatment (compromised credit card and computer chips etc). I really hope somebody can teach those bad doers a unforgettable lesson.

TheSpiceIsLife 8y ago

This is fairly analogous to my comment here: https://news.ycombinator.com/item?id=18138847 https://news.ycombinator.com/item?id=18138847 Presumably the organisations responsible for hard coding backdoors in chip designs know how to test to confirm their presence. Presumably some adversarial nation-states have moles inside these organisations > know how to remove them prior to fab. Presumably these adversaries export genuine chips to their adversaries, thereby tricking them in to thinking the backdoors made it through the fab process, and only use chips that have the backdoors removed in their own critical infrastructure. Presumably. I’ve always had this fantasy of being a hextuple agent involved in this type of deep espionage.

jsjohnst 8y ago

> as once they were closed they would have anti-tampering mechanism activated so that later it would not be possible to open the device without setting the tamper flag You didn’t specify what type of anti-tamper was used, but I wanted to jump in and say usually that means nothing. The US government intercepted packages [0] and put in back doors (removing and replacing the seals), so I’m not sure why you were so quick to dismiss state sponsored attacks by something as simple as an anti-tamper seal. You can learn how to do it yourself at most medium to large hacker conferences too (DEFCON, BlackHat, HOPE, and CCC to name a few, but there’s more with it). [0] https://www.techradar.com/news/networking/routers-storage/photos-reveal-nsa-tampered-with-cisco-router-prior-to-export-1249191 https://www.techradar.com/news/networking/routers-storage/ph...

folli 8y ago
The chinese government is probably not interested enough in credit card numbers to warrant involvement.
- lmilcin 8y ago
  It would be foolish to show your hand for few pennies.
  - jandrese 8y ago
    Might make a difference if those pennies can't be tracked back to the government.
    - malandrew 8y ago
      Yup. This would be the modern equivalent of Air America and other schemes by the CIA to raise money to operate by involving themselves in illegal activity like the drug trade.
      - jessaustin 8y ago
        One would consider those activities to have taken place in "modern" times as well...

pvarangot 8y ago

Maybe a setup to measure inertia tensor and center of mass (in that setup's axes) will be easier and I think it's what your call "measuring all angular momentums".

agumonkey 8y ago

Gosh, I would love to read everything about this. I know there are some videos about anti-tampering card readers on youtube, but not on the feedback race between hackers and security.

usrusr 8y ago

A truly sophisticated attacker would leave some easily detectable bad eggs in the stream after countering a countermeasure like the weighing.

ycmbntrthrwaway 8y ago

> We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker. You only need to measure three angular momentums, all other can be calculated. See https://en.wikipedia.org/wiki/Moment_of_inertia#Motion_in_space_of_a_rigid_body,_and_the_inertia_matrix https://en.wikipedia.org/wiki/Moment_of_inertia#Motion_in_sp... "This shows that the inertia matrix can be used to calculate the moment of inertia of a body around any specified rotation axis in the body." On the attacker side, they only need to make sure three angular momentums match.

gnode 8y ago
Maybe they're also measuring centrifugal force for rotations with axes not intersecting the centre of mass.

ramshanker 8y ago

I believe you are talking about "Rotational Moment of Inertia" about various axis, instead of "Angular Momentum". Angular Momentum of a body at rest is zero.

mycall 8y ago

Did you look into Terahertz scanning?

aarongough 8y ago

Very interesting methods you used to detect the changes! Out of curiosity: was there a reason that taking an xray of the devices was not an option? Industrial/veterinary xray machines can often be had quite cheaply...

amluto 8y ago

> We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker. You mean moment of inertia, not angular momentum. You could measure all of them! Given the moments for the three principal axes at any point, you can use the parallel axis theorem to calculate all the rest. In general, there are 10 degrees of freedom: 3 for the position of the center of mass, 3 for the axes, three for the moments, and for the the total mass. For a nicer way to count and to do the math, you have the inertia tensor at the CM (a 3-dimensional rank-2 symmetric tensor, 6 DoF) plus the location of the CM and the mass. In any event, this is a cute tampering-detection trick, but I would have started with an X-ray or CT scan.

whatshisface 8y ago
Another implication of the parallel axis theorem is that the attacker could perfectly mimic every moment of inertia by shaving plastic. They wouldn't have to know which two axes were being tested because there are only three real numbers worth of information in the system to begin with (once center of mass and total mass have been dealt with.) In the whole MOI tensor there are only six free numbers which sounds like a reasonable number of parameters to fix by adding and removing small amounts of material. What was this company doing in hiring an untrustworthy manufacturer to handle secure devices? That's playing a game you've lost from the start. Not every problem is technical!
- lmilcin 8y ago
  I honestly did not know about that. I thought that if you move any mass (remove non zero mass and place it somewhere else) there must be at least one axis which you can use to detect the change in moment of inertia.
  - ISL 8y ago
    whatshisface is correct.
  - whatshisface 8y ago
    If the mass had to all move to one other place this would be correct (the center of mass would have to change), but the attackers are able to move the mass to multiple other places.

ryandrake 8y ago

Could this be solved by a clause in the contract that specified if any randomly sampled devices came tampered with, the manufacturer did not get paid?

lmilcin 8y ago
Ah, yes, and then we would voluntarily go out of business. And the supplier will just have another customer that would not be so principled.
- ryandrake 8y ago
  Huh? This would be between you and the Chinese manufacturer. If you found s tampered unit, you don’t pay them. How should that make you go out of business?
  - pjc50 8y ago
    They need the units from somewhere, with a deadline determined by their own runway, and presumably only some of the Chinese ones were compromised. Sure, they could avoid paying, but then they have to spend months getting the production run going somewhere else .. which might also be compromised.
fma 8y ago
I'm sure the Chinese government would be willing to keep afloat companies that allow them to tamper with.

village-idiot 8y ago

At what point does it get cheaper to buy from sources that are more expensive but more trustworthy?

sonnyblarney 8y ago

Maybe time to buy your financial hardware from somewhere not china?

cyc115 8y ago
I know it's nice to blame China for everything... but it's not really the root of the problem here, supply chain management and control is.
- sonnyblarney 8y ago
  China is 100% to blame here.
  - objektif 8y ago
    Absolutely.

Gravityloss 8y ago

Why do you use those suppliers?

wnevets 8y ago

That's kinda amazing

MaxBarraclough 8y ago

So you had hard evidence that your manufacturer was vandalising your property? Couldn't you drop them?

hinkley 8y ago
The main lesson I learned from my Dad’s employer (twice) was always have two vendors. You can play them off of each other. When I did contract work I saw how powerful getting out or putting for away the checkbook can be. Large vendors ignore you if the checkbook isn’t moving. I say “twice” because they were also the biggest employer in town. I got out of there. Slim pickins for career opportunities that didn’t revolve around BigCo.

onepremise 8y ago

Seriously, why are we still outsourcing chip manufacturing to other countries? Sure it's cheaper, but we sacrifice a lot to have a society of corporate slaves build our tech. Security, core domain knowledge, capability, corporate secrets, patent rewards and enforcement, etc... All of it you throw away the minute you ship your manufacturing out of the country. I've seen enough board printing machines out there to start working on our own. As a country, we need to close this gap, more automation and capability and there will be no need to outsource circuit board printing and manufacturing. We will be much better off.

adamrezich 8y ago
It's hard to compete domestically against low prices caused by China's completely different standards for wages and human rights for laborers. Plus, here we don't like the idea of manufacturing industries being subsidized by the government (except in the case of "defense" of course...), while China obviously has no qualms with doing so.
- LargeWu 8y ago
  > except in the case of "defense" of course... Well, this thread is exactly the reason we subsidize the defense industry to make sure things are produced in the US.
TheRealPomax 8y ago
Counterpoint: even if we ignored the fact that you cannot possibly produce the volumes of chips necessary at the price necessary in your country rather than in "we don't have to acknowledge all the human rights violations" countries, why would you believe this problem goes away if chip manufacturing were done in your own country, rather than another? The moment the option of taking control of a production line of something _this important_ becomes available, your local specialized organized crime outfits will start to figure out ways to insert themselves into those production lines, learning the ins and outs, and figuring out a way to get something, anything, in there that won't be noticed but will give them a hook into millions of systems. The law does not prevent crime. It just puts a price on it. While that price is typically too high for individuals, for organizations that have no business registration to revoke, and no CEO to drag to court, it is an entirely trivial cost.

jiveturkey 8y ago

x-ray would have been easier, as others have said. but you're talking about the addition of an entire board! probably on the order of 10% of the size of the main boards. in this article, perhaps dumbed down or altered, they are talking about the addition of a single, tiny chip, too small to even be an MCU let alone have wireless capability (which BTW requires an antenna).

localhost 8y ago

For the folks commenting below that we should bring the manufacturing back to the US, why wouldn't the bad guys just start bribing American workers to insert the attack hardware into devices made here? It's not like Americans are somehow above being bribed.

assblaster 8y ago
Americans can be bribed to do anything. However, you don't really need to bribe or ask someone to betray their country and fellow citizens when it's done on Chinese soil...
lev99 8y ago
The bad actors could be brought to trial in a United States court, which is a level of deterrent not included in offshore manufacturing. If a US manufacturer was found selling tampered chips then the company itself could be held liable. This would create a general pressure to increase onsight security. There are other benefits to us based manufacturing, but you only brought up the crime aspect so I will leave the other benefits unsaid.
s73v3r_ 8y ago
It's possible, but it's much, much, much easier for an American company to hold another American company accountable when something like this happens. Instead of having to go through all those hoops, you sue the pants off the manufacturer.
pfschell 8y ago
Ignoring the abject amorality and greed that underpins Chinese culture will leave one confused like this.
- roguecoder 8y ago
  I think people here are just suggesting that the folks who believe producing things in America would avoid a culture of abject amorality and greed seem naive, at best. Even ignoring the current obvious examples, Americans frequently engage in profitable criminal behavior: https://www.businessinsider.com/mcdonalds-monopoly-game-rigged-scam-report-2018-7 https://www.businessinsider.com/mcdonalds-monopoly-game-rigg...

deleted 8y ago

[deleted]

zelon88 8y ago

Wait a minute... So your company has a Chinese equipment supplier, finds out that the supplier is tampering with your purchased equipment, and your solution is to add criteria to the incoming inspection? No wonder China keeps screwing with you guys. You aren't supposed to eat that cost! Write a PO with tons of fine print that says "We will disassembly units at random for compliance inspection. Non compliant products will be returned at the suppliers expense." And then add a clause that says ">3 non-compliance events in under # months will result in the entire PO (10 or 20 units) being returned and all contracts cancelled." I cannot believe you are getting screwed by a company you choose to do business with and yet you eat cost to ensure they aren't screwing you. Just get a new supplier! Do on-site inspections at their facility. This is nuts.

lmilcin 8y ago
There were other considerations like the fact we were actually buing it from large reputable company and what happened was that some employees were doing it with no involvement of the company. The fact is, doing any kind of hardware production in China, you have to be aware Chineese have different value system and you would not be suited doing any business if you throw tantrum at any sign of apparent dishonesty (assuming the company was involved which they could not have been as they have been the ones damaged the most). If the company does screw you (like replacing components for something cheaper) they typically will not be thinking they are doing anything wrong. They are just testing if you notice and if you do not they will say it makes no difference for you but saves them costs. The way to work is then verify everything and politely point it out. If you notice they will correct apparent mistake.
- SilasX 8y ago
  >There were other considerations like the fact we were actually buying it from large reputable company and what happened was that some employees were doing it with no involvement of the company. It's their responsibility to police for that. From the perspective of their business arrangement with you, it doesn't matter whether their left hand or right hand is evil; it's not your problem either way. Is it really hard to find alternatives? Are they really cheaper than non-Chinese sources when you account for these inspection costs?

kumarvvr 8y ago

This reminds me of a story about, IIRC, Soviet intelligence personnel determining that a photocopy machine at their consulate in the USA was bugged by the CIA by measuring it's weight and comparing that value to the standard value published by Xerox. Modern times, but same old methods of "debugging" The angular momentum stuff is innovative though.

hinkley 8y ago
Now I’m picturing a 300lbs photocopier being spun on three axes. How much tape would it take to keep doors and parts from flying off?
8note 8y ago
is measuring the moment of inertia not standard? I think any mechanical engineer would suggest it for that kind of problem (disclaimer; I've suggested it before for the same problem)

clubm8 8y ago

>measuring angular momentum on a special fixture Can you elaborate on this concept a bit? I'm not familiar with this term and the sources I looked up were pretty physics-y and out of my depth. This sounds like a very interesting and creative solution. Good lateral thinking on your part! :)

mmaunder 8y ago

This feels cargo cultish. Products drop from the sky. One day they become poisonous. You have no idea how to reproduce them locally. So you come up with hacks to make then less dangerous. We really need to get back into manufacturing if this is our brave new world.

ticviking 8y ago
The better question is how did we stop manufacturing.

amelius 8y ago

What good is angular momentum when the producer can have fluctuations in its supply chain? Yes you can see when devices are not the same, but what if that happens all the time, legitimately?

badwabbut 8y ago

I do not for one minute believe, that china, whose production floors are spotless, watched like a hawk and every product tested, did not know this was being added. They would have been able to tell themselves by weight. Yes, product weight is used for shipping manifests too. This is obfuscation of the fact, that china as a state actor has perpetrated this crime against our country period. full stop.

greglindahl 8y ago

Thanks for sharing this story, and I hope you aren't put off by the huge thread of people second-guessing your competence. We need more of your kind of story.

ndesaulniers 8y ago

Was the secret service contacted? Card skimmers are a big no no. Family friend works for Dept of weights and measures and finds skimmers all the time on gas pumps. Scary stuff.