8 ms·

From Apple’s actual iCloud security document: > Each file is broken into chunks and encrypted by iCloud using AES-128 and a key derived from each chunk’s conte

From Apple’s actual iCloud security document: > Each file is broken into chunks and encrypted by iCloud using AES-128 and a key derived from each chunk’s contents that utilizes SHA-256. The keys and the file’s metadata are stored by Apple in the user’s iCloud account. The encrypted chunks of the file are stored, without any user-identifying information, using third-party storage services, such as S3 and Google Cloud Platform. From what I’ve heard, Apple’s services run on their own cloud platform called Pie [0]. It sounds like this platform probably abstracts away whatever storage service is used, allowing Apple to use whatever fits their requirements. [0] - https://9to5mac.com/2016/10/06/report-unified-cloud-services-backend-pie-platform/ https://9to5mac.com/2016/10/06/report-unified-cloud-services...

flixic 9y ago
[Apple] Pie in the Sky for Cloud platform. Clever name, looks like clever tech.
- exelius 9y ago
  Every fortune 50 company has a similar system at this point; you tie it in with your enterprise architecture platform so you can go directly from design to infrastructure using tested patterns. Pretty much table stakes in large enterprise at this point; and as usual IBM / SAP / Oracle / SoftwareAG cabal have bought up everything that matters.
- oil7abibi 9y ago
  Idk, seems like that would be a pretty standard abstraction layer to implement at a large company.
  - ksec 9y ago
    Sorry if this is an naive question. If every large companies are doing it, why aren't there a standard open source abstraction library available?
    - deleted 9y ago
      [deleted]
    - tempay 9y ago
      Often the reason to open source is to get developer mindshare (tensorflow) or in the hope of gaining traction to compete against existing closed source tech (opencompute).
    - cobbzilla 9y ago
      I used Jclouds [1] a few years back to do this. While it's very powerful, it's also fairly complex (abstracts not just storage but also compute/etc). At the time, I remember wanting a simpler abstraction layer over it, and ended up building my own. Maybe something like that exists now, I'm not sure. The devil is in the details of how each provider offers vendor-specific things and you want to take advantage of them. For example Reduced-Redundancy storage is iirc an Amazon-specific offering, or if others offer it, it's probably under different SLA terms/measurements. This rapidly breaks many generic abstractions; maybe this is why everyone ends up writing their own little shim layer for their situation. In some sense it reminds me a bit of building database-connection-pools in the 90's, before they were really standardized everyone rolled their own and learned all the awful lessons about reference counting along the way. Then along came ODBC, then JDBC, and things were so much easier because you only had to deal with one API, and the databases would conform to their side of it. So I think, isn't that what OpenStack (or something?) is supposed to be for cloud services? But whoa, the depth and complexity of these services far exceeds that of a 90's database. It will take a while -- but over time and with patterns of common use well established, a stable base of standard APIs will abstract away most differences, making things so much nicer. I can dream. [1] https://jclouds.apache.org/ https://jclouds.apache.org/
    - rsync 9y ago
      "Sorry if this is an naive question. If every large companies are doing it, why aren't there a standard open source abstraction library available?" I think Tahoe LAFS is the open source solution for this. https://tahoe-lafs.org/trac/tahoe-lafs https://tahoe-lafs.org/trac/tahoe-lafs https://en.wikipedia.org/wiki/Tahoe-LAFS https://en.wikipedia.org/wiki/Tahoe-LAFS
    - 9y ago
  - duxup 9y ago
    Yeah I don't know how many companies do it, but it seems like a process that in some form is a good best practice.
    - EpicEng 9y ago
      The problem there being that small companies get fooled into thinking they need it and spend a bunch of time abstracting away (and complicating!) a service layer they'll likely never change.
  - toephu2 9y ago
    Yeah, as if you could whip this up in an afternoon.
    - zodPod 9y ago
      I didn't realize that was a requirement for something being standard...
- oliTwist23 9y ago
  There used to be a project called Geneva. Not sure if this is same.
kudokatz 9y ago
> Each file is broken into chunks and encrypted by iCloud using AES-128 and a key derived from each chunk’s contents this is exactly the strategy described at the Google Cloud Summit 2017 relating to how security is managed for their platform ... so it lines up
- deleted 9y ago
  [deleted]
- delroth 9y ago
  Disclaimer: I work at Google, not on cloud. I think this is just coincidence though. Apple most likely performs their own layer of encryption on top of what Google already does. They protect against two different attack scenarios: Google doesn't want plaintext data to show up on disks. Apple doesn't want plaintext data to be accessible by anyone at Google (or any other 3rd party provider).
greensoap 9y ago
The encryption technique sounds like convergent encryption. Convergent encryption is a fairly standard way storing encrypted data that might duplicate across users. It allows for deduping of data while in an encrypted state. https://en.wikipedia.org/wiki/Convergent_encryption https://en.wikipedia.org/wiki/Convergent_encryption
- matt_wulfeck 9y ago
  Doesn’t this in some way betray the encryption? For example, if multiple users had illegal.csv in their accounts, getting access to one user will allow you to prosecute any of them without decrypting their data because they will all share the same sha?
  - simias 9y ago
    It is (as the Wikipedia article points out). It's basically equivalent to storing passwords hashed without a salt, it makes makes it obvious when a password is reused, or in this case the same file stored twice. That means that the provider always knows if a file is shared across multiple users (even if they don't know what the file is) and given a cleartext file they can always check if somebody has it stored on the service. It's not ideal at all if you want good privacy. SpiderOak is explicitly supposed not to do that[1] to avoid these issues, however I refuse to recommend them until they finally decide to release an open source client and 2FA support so caveat emptor. Anyway, in this case it's irrelevant because Apple has access to the keys anyway, it's only supposed to prevent the third party (Google Cloud in this case) from having access to the files. [1] https://spideroak.com/resources/encryption-white-paper https://spideroak.com/resources/encryption-white-paper in the "Data Deduplication" section.
    - Someone1234 9y ago
      It should be pointed out that this "security flaw" is the only way these cloud file storage platforms are cost effective. Since it allows the same anonymous blob stored by several dozen users to only take up one unit of space instead of one unit per user. This is particularly space saving if a particular file type was chunked cleverly so that static parts of the file's structure were stored away from dynamic parts (Microsoft's Office XML formats for example could definitely be split this way).
smrtinsert 9y ago
It's important to abstract it away so that the CEO can flip a switch and suddenly they're at a new cloud provider. Don't make me laugh.

flixic 9y ago

[Apple] Pie in the Sky for Cloud platform. Clever name, looks like clever tech.

exelius 9y ago
Every fortune 50 company has a similar system at this point; you tie it in with your enterprise architecture platform so you can go directly from design to infrastructure using tested patterns. Pretty much table stakes in large enterprise at this point; and as usual IBM / SAP / Oracle / SoftwareAG cabal have bought up everything that matters.
oil7abibi 9y ago
Idk, seems like that would be a pretty standard abstraction layer to implement at a large company.
- ksec 9y ago
  Sorry if this is an naive question. If every large companies are doing it, why aren't there a standard open source abstraction library available?
  - deleted 9y ago
    [deleted]
  - tempay 9y ago
    Often the reason to open source is to get developer mindshare (tensorflow) or in the hope of gaining traction to compete against existing closed source tech (opencompute).
  - cobbzilla 9y ago
    I used Jclouds [1] a few years back to do this. While it's very powerful, it's also fairly complex (abstracts not just storage but also compute/etc). At the time, I remember wanting a simpler abstraction layer over it, and ended up building my own. Maybe something like that exists now, I'm not sure. The devil is in the details of how each provider offers vendor-specific things and you want to take advantage of them. For example Reduced-Redundancy storage is iirc an Amazon-specific offering, or if others offer it, it's probably under different SLA terms/measurements. This rapidly breaks many generic abstractions; maybe this is why everyone ends up writing their own little shim layer for their situation. In some sense it reminds me a bit of building database-connection-pools in the 90's, before they were really standardized everyone rolled their own and learned all the awful lessons about reference counting along the way. Then along came ODBC, then JDBC, and things were so much easier because you only had to deal with one API, and the databases would conform to their side of it. So I think, isn't that what OpenStack (or something?) is supposed to be for cloud services? But whoa, the depth and complexity of these services far exceeds that of a 90's database. It will take a while -- but over time and with patterns of common use well established, a stable base of standard APIs will abstract away most differences, making things so much nicer. I can dream. [1] https://jclouds.apache.org/ https://jclouds.apache.org/
  - rsync 9y ago
    "Sorry if this is an naive question. If every large companies are doing it, why aren't there a standard open source abstraction library available?" I think Tahoe LAFS is the open source solution for this. https://tahoe-lafs.org/trac/tahoe-lafs https://tahoe-lafs.org/trac/tahoe-lafs https://en.wikipedia.org/wiki/Tahoe-LAFS https://en.wikipedia.org/wiki/Tahoe-LAFS
  - 9y ago
- duxup 9y ago
  Yeah I don't know how many companies do it, but it seems like a process that in some form is a good best practice.
  - EpicEng 9y ago
    The problem there being that small companies get fooled into thinking they need it and spend a bunch of time abstracting away (and complicating!) a service layer they'll likely never change.
- toephu2 9y ago
  Yeah, as if you could whip this up in an afternoon.
  - zodPod 9y ago
    I didn't realize that was a requirement for something being standard...
oliTwist23 9y ago
There used to be a project called Geneva. Not sure if this is same.

kudokatz 9y ago

> Each file is broken into chunks and encrypted by iCloud using AES-128 and a key derived from each chunk’s contents this is exactly the strategy described at the Google Cloud Summit 2017 relating to how security is managed for their platform ... so it lines up

deleted 9y ago
[deleted]
delroth 9y ago
Disclaimer: I work at Google, not on cloud. I think this is just coincidence though. Apple most likely performs their own layer of encryption on top of what Google already does. They protect against two different attack scenarios: Google doesn't want plaintext data to show up on disks. Apple doesn't want plaintext data to be accessible by anyone at Google (or any other 3rd party provider).

greensoap 9y ago

The encryption technique sounds like convergent encryption. Convergent encryption is a fairly standard way storing encrypted data that might duplicate across users. It allows for deduping of data while in an encrypted state. https://en.wikipedia.org/wiki/Convergent_encryption https://en.wikipedia.org/wiki/Convergent_encryption

matt_wulfeck 9y ago
Doesn’t this in some way betray the encryption? For example, if multiple users had illegal.csv in their accounts, getting access to one user will allow you to prosecute any of them without decrypting their data because they will all share the same sha?
- simias 9y ago
  It is (as the Wikipedia article points out). It's basically equivalent to storing passwords hashed without a salt, it makes makes it obvious when a password is reused, or in this case the same file stored twice. That means that the provider always knows if a file is shared across multiple users (even if they don't know what the file is) and given a cleartext file they can always check if somebody has it stored on the service. It's not ideal at all if you want good privacy. SpiderOak is explicitly supposed not to do that[1] to avoid these issues, however I refuse to recommend them until they finally decide to release an open source client and 2FA support so caveat emptor. Anyway, in this case it's irrelevant because Apple has access to the keys anyway, it's only supposed to prevent the third party (Google Cloud in this case) from having access to the files. [1] https://spideroak.com/resources/encryption-white-paper https://spideroak.com/resources/encryption-white-paper in the "Data Deduplication" section.
  - Someone1234 9y ago
    It should be pointed out that this "security flaw" is the only way these cloud file storage platforms are cost effective. Since it allows the same anonymous blob stored by several dozen users to only take up one unit of space instead of one unit per user. This is particularly space saving if a particular file type was chunked cleverly so that static parts of the file's structure were stored away from dynamic parts (Microsoft's Office XML formats for example could definitely be split this way).

smrtinsert 9y ago

It's important to abstract it away so that the CEO can flip a switch and suddenly they're at a new cloud provider. Don't make me laugh.