5 ms·

That's quite scary. I wonder if something like this is possible at Google or Microsoft or Yahoo. Even if multiple people need to approve that kind of access, it

That's quite scary. I wonder if something like this is possible at Google or Microsoft or Yahoo. Even if multiple people need to approve that kind of access, it must be possible to socially overcome those barriers (via influence, bribery, etc.) if the right actors can be identified. It would be preferable to have control over this from the user-side.

ajhurliman 6y ago
This is the idea of local-first software[0]. Imagine you owning your own database and the only thing you get when you go to a website is the software and none of the data (it accesses your database instead). Projects like Textile[1] are building out tools that could help with that sort of project. I really hope it takes off, the cloud sort of freaks me out. [0] https://www.inkandswitch.com/local-first.html https://www.inkandswitch.com/local-first.html [1] http://textile.io/ http://textile.io/
- yrgulation 6y ago
  Either that or each user’s records are encrypted using their own password and forgot password recovery question/s answers. Here is an example of how it can be done: https://security.stackexchange.com/questions/30193/encrypting-user-data-using-password-and-forgot-my-password https://security.stackexchange.com/questions/30193/encryptin... Although this approach still requires trust that service providers do indeed encrypt user data.
- Scene_Cast2 6y ago
  So the core problem is that when something is running on someone else's server, you can't even verify what they're running. E2E encryption avoids that by not trusting whatever is running on the servers Local DBs avoid that by not giving up local data. However, it would be quite interesting to have a way to remotely know that a certain service is running the code you think it's running.
  - jaywalk 6y ago
    > it would be quite interesting to have a way to remotely know that a certain service is running the code you think it's running. It sure would, and it would be a hell of a discovery if someone could come up with it. Because I sure can't think of a way that I can't easily debunk.
    - Scene_Cast2 6y ago
      Intel has done some work on this front that can hypothetically be used this way, but I wouldn't say it's practical, for various reasons.
  - DSingularity 6y ago
    You mean like intel TXT remote attestation?
    - Scene_Cast2 6y ago
      The trick here would be integrating that to play nicely with load balancers, REST-style APIs, and reducing overhead from establishing yet another secure connection.
- hezag 6y ago
  Related: Solid Project - HN discussion: https://news.ycombinator.com/item?id=25989698 https://news.ycombinator.com/item?id=25989698 - A great article about the project: https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-fosters-sustainable-innovation/ https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-...
  - Jugurtha 6y ago
    Slightly related, we're working with a similar philosophy. As a machine learning consultancy that has done many learning projects for enterprise, we're building our machine learning operations, "MLOps", platform (https://iko.ai https://iko.ai) to simplify our work. However, what we're doing is working from the architecture level to have as little and preferrably no sensitive information on our service. We're architecting it so that you give us specific access to deploy on your cluster, and everything happens there: the notebook servers are there, your data is where you choose to put it, your training jobs are there, your experiments are tracked there. Your models are deployed there. I have a saying that the platform should be able to run on a Raspberry PI. One of my personal pet peeves working with the team is to be able to disappear without impacting them, and it has become the same with our platform: it must be able to disappear users having to scramble to exfiltrate or export their work or data from our infrastructure, because it simply is not there.
sn_master 6y ago
It is. For example, machine learning teams at Microsoft run software that reads all email content on Exchange servers. There are guardrails to make sure the engineers don't gain access to the data themselves, but there are accidental slip ups from time to time, and certainly a motivated engineer can always find a way to peek at the data of any inbox.
- fractionalhare 6y ago
  Yeah. That could be resolved if the ML teams only had access to the aggregated, anonymized data or the output of the models. And if a privileged access token (for example, the one the model training flow ostensibly uses) is logged as querying specific subsets of the raw data or ferrying it out of band, that should throw an immediate alarm with an audit trail.
  - deleted 6y ago
    [deleted]
ceejayoz 6y ago
Not just possible, it happens. Google: https://gawker.com/5637234/gcreep-google-engineer-stalked-teens-spied-on-chats https://gawker.com/5637234/gcreep-google-engineer-stalked-te... Facebook: https://www.theguardian.com/technology/2018/may/02/facebook-engineer-fired-alleged-stalker-tinder https://www.theguardian.com/technology/2018/may/02/facebook-... The NSA: https://www.reuters.com/article/us-usa-surveillance-watchdog/nsa-staff-used-spy-tools-on-spouses-ex-lovers-watchdog-idUSBRE98Q14G20130927 https://www.reuters.com/article/us-usa-surveillance-watchdog... These are presumably just the tip of the iceberg of people dumb enough to get caught.
- redis_mlc 6y ago
  Yahoo historically (for decades) monitors customer support IC staff for application-level access abuse to user account data, and does investigations for misuse. Mgmt. takes that seriously, so the Yandex scenario mentioned in the article wouldn't happen for long. However, as at all companies, engineers have alternate server-level access to row-level data, otherwise nobody could troubleshoot internal systems. Yahoo is divided into 50+ engineering silos, so that access is very diffused. So far, so good. But in the case of Yahoo, the govt. does kernel-level keyword sniffing on email servers. (AFAIK that's unique to Yahoo. Never even heard of that for FAANG.) Yahoo was also pwned for a few years: https://www.theregister.com/2018/04/24/yahoo_fined_35m/ https://www.theregister.com/2018/04/24/yahoo_fined_35m/ https://www.theregister.com/2016/10/04/yahoo_was_nsa_stooge/ https://www.theregister.com/2016/10/04/yahoo_was_nsa_stooge/ Source: worked there.
- tdeck 6y ago
  That Google link is from 2010. Access controls at Google are much stricter now.
- fingerlocks 6y ago
  Did you even read the article about Facebook? Some dude got fired over a screenshot of a out-of-context lame joke text message. It was a weak allegation

ajhurliman 6y ago

This is the idea of local-first software[0]. Imagine you owning your own database and the only thing you get when you go to a website is the software and none of the data (it accesses your database instead). Projects like Textile[1] are building out tools that could help with that sort of project. I really hope it takes off, the cloud sort of freaks me out. [0] https://www.inkandswitch.com/local-first.html https://www.inkandswitch.com/local-first.html [1] http://textile.io/ http://textile.io/

yrgulation 6y ago
Either that or each user’s records are encrypted using their own password and forgot password recovery question/s answers. Here is an example of how it can be done: https://security.stackexchange.com/questions/30193/encrypting-user-data-using-password-and-forgot-my-password https://security.stackexchange.com/questions/30193/encryptin... Although this approach still requires trust that service providers do indeed encrypt user data.
Scene_Cast2 6y ago
So the core problem is that when something is running on someone else's server, you can't even verify what they're running. E2E encryption avoids that by not trusting whatever is running on the servers Local DBs avoid that by not giving up local data. However, it would be quite interesting to have a way to remotely know that a certain service is running the code you think it's running.
- jaywalk 6y ago
  > it would be quite interesting to have a way to remotely know that a certain service is running the code you think it's running. It sure would, and it would be a hell of a discovery if someone could come up with it. Because I sure can't think of a way that I can't easily debunk.
  - Scene_Cast2 6y ago
    Intel has done some work on this front that can hypothetically be used this way, but I wouldn't say it's practical, for various reasons.
- DSingularity 6y ago
  You mean like intel TXT remote attestation?
  - Scene_Cast2 6y ago
    The trick here would be integrating that to play nicely with load balancers, REST-style APIs, and reducing overhead from establishing yet another secure connection.
hezag 6y ago
Related: Solid Project - HN discussion: https://news.ycombinator.com/item?id=25989698 https://news.ycombinator.com/item?id=25989698 - A great article about the project: https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-fosters-sustainable-innovation/ https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-...
- Jugurtha 6y ago
  Slightly related, we're working with a similar philosophy. As a machine learning consultancy that has done many learning projects for enterprise, we're building our machine learning operations, "MLOps", platform (https://iko.ai https://iko.ai) to simplify our work. However, what we're doing is working from the architecture level to have as little and preferrably no sensitive information on our service. We're architecting it so that you give us specific access to deploy on your cluster, and everything happens there: the notebook servers are there, your data is where you choose to put it, your training jobs are there, your experiments are tracked there. Your models are deployed there. I have a saying that the platform should be able to run on a Raspberry PI. One of my personal pet peeves working with the team is to be able to disappear without impacting them, and it has become the same with our platform: it must be able to disappear users having to scramble to exfiltrate or export their work or data from our infrastructure, because it simply is not there.

sn_master 6y ago

It is. For example, machine learning teams at Microsoft run software that reads all email content on Exchange servers. There are guardrails to make sure the engineers don't gain access to the data themselves, but there are accidental slip ups from time to time, and certainly a motivated engineer can always find a way to peek at the data of any inbox.

fractionalhare 6y ago
Yeah. That could be resolved if the ML teams only had access to the aggregated, anonymized data or the output of the models. And if a privileged access token (for example, the one the model training flow ostensibly uses) is logged as querying specific subsets of the raw data or ferrying it out of band, that should throw an immediate alarm with an audit trail.
- deleted 6y ago
  [deleted]

ceejayoz 6y ago

Not just possible, it happens. Google: https://gawker.com/5637234/gcreep-google-engineer-stalked-teens-spied-on-chats https://gawker.com/5637234/gcreep-google-engineer-stalked-te... Facebook: https://www.theguardian.com/technology/2018/may/02/facebook-engineer-fired-alleged-stalker-tinder https://www.theguardian.com/technology/2018/may/02/facebook-... The NSA: https://www.reuters.com/article/us-usa-surveillance-watchdog/nsa-staff-used-spy-tools-on-spouses-ex-lovers-watchdog-idUSBRE98Q14G20130927 https://www.reuters.com/article/us-usa-surveillance-watchdog... These are presumably just the tip of the iceberg of people dumb enough to get caught.

redis_mlc 6y ago
Yahoo historically (for decades) monitors customer support IC staff for application-level access abuse to user account data, and does investigations for misuse. Mgmt. takes that seriously, so the Yandex scenario mentioned in the article wouldn't happen for long. However, as at all companies, engineers have alternate server-level access to row-level data, otherwise nobody could troubleshoot internal systems. Yahoo is divided into 50+ engineering silos, so that access is very diffused. So far, so good. But in the case of Yahoo, the govt. does kernel-level keyword sniffing on email servers. (AFAIK that's unique to Yahoo. Never even heard of that for FAANG.) Yahoo was also pwned for a few years: https://www.theregister.com/2018/04/24/yahoo_fined_35m/ https://www.theregister.com/2018/04/24/yahoo_fined_35m/ https://www.theregister.com/2016/10/04/yahoo_was_nsa_stooge/ https://www.theregister.com/2016/10/04/yahoo_was_nsa_stooge/ Source: worked there.
tdeck 6y ago
That Google link is from 2010. Access controls at Google are much stricter now.
fingerlocks 6y ago
Did you even read the article about Facebook? Some dude got fired over a screenshot of a out-of-context lame joke text message. It was a weak allegation