10 ms·
Report: Apple building its own servers to prevent snooping
- dang 11y agoUrl changed from http://daringfireball.net/linked/2016/03/23/apple-data-centers http://daringfireball.net/linked/2016/03/23/apple-data-cente..., which points to this.
- samstave 11y ago"Apple has long suspected that servers it ordered from the traditional supply chain were intercepted during shipping, with additional chips and firmware added to them by unknown third parties in order to make them vulnerable to infiltration" Wow, so this is very interesting given, pretty much, everything that has been going on. Will apple actually be the bastion of freedom (both in market and privacy) that the US supposedly stands for?? Google makes its own machines, as does facebook (actually more interested in FB's fiber switches/routers, but thats beside the point).... But Apple has made "servers" for years... I guess they didnt consume them in their own DCs?? So I basically take it that they are effectively joining their take on Open Compute (mobos that can be mounted in controlled environments, where controlled now also means they can ID if anything was modded/changed in shipping?) EDIT: I would really like to know how long "long" is from "apple long suspected".... I was informed of NSA back-doors in Cisco gear in 1997 - WTF is Cisco's stance on any of this -- I haven't heard anything from them at all (or I missed anything they said)
- seanp2k2 11y agoXserves were replaced a few years ago. The current "replacement" is the Mac Pro. I highly doubt that all of iCloud runs on racks filled with the little cylinders. https://en.wikipedia.org/wiki/Xserve https://en.wikipedia.org/wiki/Xserve http://readwrite.com/2014/08/26/apple-icloud-amazon-web-services-hosting/ http://readwrite.com/2014/08/26/apple-icloud-amazon-web-serv...
- protomyth 11y agoIf they were really paranoid they could build a new Xserve powered by an offshoot of their A series ARM chips. To help defray costs, they could actually sell the thing.
- npunt 11y agoThat'd be an awesome extension of the resources they invested, but it runs the risk of designs serving too many masters (like the F35 Joint Strike Fighter trying to add VTOL, making the non-VTOL thing way too heavy), and the A-series probably doesn't have enough cores for the unit price to work well for these services. What Intel is doing with Xeon Phi and the 18-core Xeons is the way things are going I think.
- protomyth 11y agoI think they could handle a separate line of chips. Plus, a lot of folks are going the ARM route in server-land.
- ryao 11y agoIf their A-series chips have protected internal data paths and ECC, they could build microservers around them. The A9 already has PCI-E lanes that could be used for networking and storage.
- rdancer 11y agoIf for every F35 they made three of each, with different capabilities, and ceremonially dumped the two without the desired capabilities out in the ocean, it would still be cheaper overall. Which is hopefully what Apple would do. It's not like they lack talent or organizational ability to employ a few more chip designers. Or a few dozen.
- acdha 11y agoThe A9 does have respectable performance and great thermal characteristics. Rather than compete head to head with Intel, who are very very good a general purpose CPUs, it'd be interesting to see a cloud-optimized “server” which is actually a 1U cluster of storage processors – tweaked A9, as much flash as they can fit, etc. – so you'd have capacity to do things like encrypt everything at rest and in transit and apply distributed database techniques to everything because you'd have a relatively huge amount of processing capacity very close to each block of storage and they control the entire software stack. Basically like what Oracle does with Exadata without fitting a yacht into the purchase price.
- jonathankoren 11y agoSounds like NSA's Tailored Access Operations[0], and other equivalent agencies. When you're a big enough target, state actors bring the full force of the state's espionage tools to bear. [0] http://www.pbs.org/wgbh/frontline/article/how-the-nsas-secret-elite-hacking-unit-works/ http://www.pbs.org/wgbh/frontline/article/how-the-nsas-secre...
- samstave 11y agoExactly! How many machines have you purchased in your career? I've purchased THOUSANDS - which of these have been intercepted and compromised? I have no clue.
- Spooky23 11y agoI don't know about spy agency compromise, but I have definately caught suppliers "accidentally" sticking counterfeit memory into devices.
- avn2109 11y ago>> "...which of these have been...compromised...? ~All of them, though likely without bothering to intercept your specific hardware, since you were relatively unimportant. Instead, the breaches were built into commodity hardware far upstream of you, for fully general attacks.
- iam-TJ 11y agoIt occurs to me if the company is that big, or that big of a target, it can manage to collect its servers directly from the manufacturer/supplier rather than rely on a third unrelated party for transportation.
- joering2 11y agoSo what? How do you know Dell/HP/Whoever has not been forced by NSL letters to implement backdoors.
- 11y ago
- JustSomeNobody 11y agoIntercepted during shipping? How long would an operation like that take? And can't a company with Apple's clout ask the supplier for a spec sheet that lists all the chips and a diagram showing the location of said chips?
- samstave 11y agoOf course they can, which is why perhaps apple "long suspected"... maybe shipping times were fucked and apple noticed - but my question is WHEN did apple notice???
- jkestner 11y agoYou sound incredulous. It's a thing. http://www.pcworld.com/article/2083300/report-nsa-intercepts-computer-deliveries-to-plant-spyware.html http://www.pcworld.com/article/2083300/report-nsa-intercepts...
- ryao 11y agoThe Intel chipsets even have an embedded device with complete access to everything, its own firmware and its own integrated flash where malware can be flashed by anyone with Intel's internal documentation and the private RSA key corresponding to the public key that is etched into the hardware: https://libreboot.org/faq/#intelme https://libreboot.org/faq/#intelme
- JustSomeNobody 11y agoA bit. So are they going to run these in another country? Because Congress is pretty idiotic when it comes to technology and they can pass laws to force Apple (and others) to allow snooping.
- mc32 11y agoThis doesn't make much sense. Even if apple made their own, unless they will be shipping with their own fleet of delivery trucks with armed guards not on the take, how would this differ from shipping dell machines, or if they do go this route, why not use their own guarded shipping for Dells and netapp stuff? Not only that, but they can confirm the firmware after receiving the equipment, unless they suspect the CPUs machine code, but then who will they get their CPUs from? And besides, it's not like they'll be tearing out all the netapp, dells and Cisco --everyone even amazon have them all, from what I hear.
- randall 11y agoIt's a lot easier to diff if you invented the originals.
- mtgx 11y agoThey could do stuff like this for their own servers: https://www.kickstarter.com/projects/designshift/orwl-the-first-open-source-hardened-computer https://www.kickstarter.com/projects/designshift/orwl-the-fi... But they may want to build their own chips, too, because they probably shouldn't trust Intel (they do some of that already, but they may need more powerful chips for their datacenters).
- sna1l 11y agoI have a buddy who was super high up in a telecom back in the late 90s/early 00s and he mentioned that all the big telecoms basically voluntarily were creating traffic backdoors for the government. When he mentioned it, he didn't even blink. It was eerie.
- samstave 11y agoEchelon. It is real. It has been around forever.
- tetheno 11y agoI don't know how Apple's servers security fares now, but I took a quick look 4 years ago and it was notoriously bad. They were responsive and solved the exploitable bugs soon after I notified them. No bounties though. If their software and network security is similar now.. then they should spend resources there rather than care too much about modified hardware by a governmental agency.
- hrktb 11y agoWouldn't it be completely separate teams ? I can't imagine the hardware team's efforts taking away engineers from teams handling software security.
- mc32 11y agoAn issue at very large companies like Apple is that while they can be superb in one area or in one core product does not translate into being adept in other areas under different teams with different management styles, demands budgets, etc. So unless Apple are going to put their best and poach other industry bests, their result will likely not deliver on premise or promise.
- lostlogin 11y agoJudging by their rhetoric and their recent win (can it be called that?), there is certainly widespread interest in them locking themselves down. They would look a little foolish if not.
- tetheno 11y agoCertainly they are different teams. My comment just meant that I needed 1 hour to find a way to get a shell in a couple of their servers. That is more worrisome to me than the NSA snooping around. And I'm not a great pentester, I probably wouldn't find a bug in Google even if I spent a couple weeks.
- ryao 11y agoThey are HP built Intel systems running Linux.
- ikeboy 11y agoIn theory, does a backdoored firmware run slower? If yes, then can you detect a backdoor by building one yourself and benchmarking? Or are the margins of error on repeated benchmarks larger than any performance hit due to a backdoor, or can something be backdoored without any performance hit? What about power consumption?
- strooper 11y agoPractically, backdoored firmware doesn't necessarily run services and communicates to the mothership all the time, nor does it communicate using conventional protocols. It creates invisible hole in the system beyond OS's reach to detect and repair it. Furthermore, the hardware (or chip based) backdoor is way more sophisticated, typically have been in use in the servers of foreign governments of interest. This chips can be used to take full control of a server, and the whole network when places strategically. In another note, hard RTOS are not used in servers, so performance cannot be compared with benchmarking tools.
- IncRnd 11y ago>> In another note, hard RTOS are not used in servers, so performance cannot be compared with benchmarking tools. It may seem surprising, but performance is easily determined by side channel attacks. https://www.google.com/search?q=blackhat+side+channel+web+timing https://www.google.com/search?q=blackhat+side+channel+web+ti...
- rdancer 11y agoIt runs at hardware speeds, and only some of the time, so not really. You can perform timing attacks, and see if you're running under a modified hypervisor, for example. But if you have a sniffer chip on the PCI bus or Northbridge, you won't be able to see it. Power consumption has been done as well, but you really need to know what it is you're looking for, and if it is just dormant for most of the time, how are you going to test for that?
- ryao 11y agoApple might want to talk to IBM about OpenPOWER. The POWER8 chips and chipsets lack known internal micro controllers that have their own flash unlike Intel's chips that have the Intel management engine. Consequently, there is nothing there to flash with malware. Also, all of the system firmware is open source. That is the motivation behind the Talos workstation board, provided that they get enough interest for an initial run: https://raptorengineeringinc.com/TALOS/prerelease.php https://raptorengineeringinc.com/TALOS/prerelease.php If Apple is concerned about tampering enroute, they could have the flash chips for the system firmware provided separately by a trusted party and transported like a bank shipment. Then flash and install them at the datacenter. That should thwart adversaries who cannot do their own manufacturing runs of modified versions of the chips, which is just about everyone. I guess the manufacturer/fabrication plant could do a custom compromised chip, but given that the costs involved are prohibitive, I doubt that would happen. Apple could do the same with the firmware and flash for every other component in their datacenter that has a microprocessor such as the hard drives, the NICs, etcetera. They are large enough that part manufacturers would likely turn over the source code for their firmware in order to secure their business along with anything else that they need/want.
- avn2109 11y ago>> "...transported like a bank shipment..." If you think the NSA/state level adversaries have not penetrated bank shipments, you are dangerously naive. Instead, maybe transport it like the CIA moves the communication hardware for its quiet rooms, or like the Russians move the bugs destined for US embassies. Apple is interesting because it's one of the few organizations that could plausibly do this.
- RRRA 11y agoWhen will we get secure PC architecture for the consumer market?
- jen729w 11y agoWhen it's in someone's financial interest to build it. Which is to say, never.
- meddlepal 11y agoNever.
- ryao 11y agoWhen consumers start paying for things like Talos: https://raptorengineeringinc.com/TALOS/prerelease.php https://raptorengineeringinc.com/TALOS/prerelease.php
- beeboop 11y agohttps://www.crowdsupply.com/sutajio-kosagi/novena https://www.crowdsupply.com/sutajio-kosagi/novena as close as we have, at least in terms of verifiabily uncompromised.
- Animats 11y agoIs Apple saying anything about what they found? This is a big deal. Who put something in their servers? NSA? The PLA? Samsung? Did they have any strange chips analyzed? There are companies that can take an IC apart and see what's inside.[1][2] It's not cheap, but Apple could afford it. [1] http://www.siliconinvestigations.com/ http://www.siliconinvestigations.com/ [2] http://www.istgroup.com/english/3_service/03_01_detail.php?MID=43&SID=86&ID=251 http://www.istgroup.com/english/3_service/03_01_detail.php?M...
- beefsack 11y agoNo company is static, what will Apple look like in 10 years? What would happen with my data they recorded today should company policy take a shift? I'm quite uneasy regardless of who holds my data.
- uptown 11y ago"Apple has long suspected that servers it ordered from the traditional supply chain were intercepted during shipping, with additional chips and firmware added to them by unknown third parties in order to make them vulnerable to infiltration, according to a person familiar with the matter." I feel like the entire world has gone insane, and every boundary is being pushed to its limits ... and then pushed beyond those limits. Where does this end?
- ryao 11y agoThe world was that way long before you realized it.
- rdancer 11y agoNo, this is post-9/11. It only started with the PATRIOT Act. I vividly remember the Clipper chip debates in Congress. This is that on steroids, with no disclosure what-so-ever. In the communist block, that was different, of course. But in the West, you used to laugh and sneer at Stasi and Soviet "papers, please!".
- anthk 11y agoPoor delusional.
- adventured 11y agoThe only reason this wasn't happening in the days of J Edgar Hoover, was due to technological limitations. Echelon dates to the late 1960s. COINTELPRO went on for two decades starting from 1956. Privacy and rights abuses have been rampant among the three letter agencies since their originations. Today, they can scale the privacy invasion. That's the sole difference from their side. Previously they'd just violate someone's rights and go about their business, good luck proving it or fighting it at a small scale as a specific target of one of these agencies. https://en.wikipedia.org/wiki/ECHELON https://en.wikipedia.org/wiki/ECHELON https://en.wikipedia.org/wiki/COINTELPRO https://en.wikipedia.org/wiki/COINTELPRO
- onRoadAgain23 11y agoIf this is a trend I guess HP wishes it would never have so readily worked with the NSA bugging servers.
- benmmurphy 11y agoif the NSA is doing this in the US against a US company isn't this illegal?
- abysmal_cynic 11y ago"Legality", in this context, went out the window long ago. Power relentlessly weaponises "legality". Power uses it to attack its enemies and to shield the unethical activities it wishes to protect.
- siliconviking 11y agoWhy not just contribute a few designs or some open source software and join OCP? http://opencompute.org/ http://opencompute.org/ Last time I checked, Apple was not in the business of making data center equipment, so it's not like they would give up IP that is central to their business model.
- abysmal_cynic 11y agoLike the FBI vs Apple case, this looks like marketing hype to me. Apple is increasingly selling on "security" they can't deliver. (I mean, I'm sure they'll try to action something, but we all know security is a compromise not a guarantee).
- smoser 11y ago“[23:06] <ATPtipster> we've gotten Cisco equipment, Supermicro servers, and Seagate hard drives that have been tampered with [by the NSA]”
- sickbeard 11y agoSo Apple is supposed to safeguard our data now? If you really want privacy don't expect big corp to keep it for you.
- massemphasis 11y agoThis is one of the faultier parts of having to answer to share holders. Apple is finding it more difficult to justify having their own entire supply chain rather than relying on other companies (i.e. Intel, IBM, Samsung, etc...) If they didn't make that horrible mistake of their recent multi-billion dollar purchases, they'd have extra billion atleast towards a fab plant. I'm laughing at Apple right now. Idiots. You Apple board... are stupid.
- draw_down 11y agoWhat? Why would they care about a billion dollars? They're sitting on around $200 billion cash.
- massemphasis 11y agoThey are seen as throwing away cash, and throwing away 1% of cash holdings is very bad.