7 ms·

Very naive question. Suppose I offer a service (e.g. Have I Been Pwned) and I want my users to trust my service not to store/share their data. I understand that

Very naive question. Suppose I offer a service (e.g. Have I Been Pwned) and I want my users to trust my service not to store/share their data. I understand that open sourcing the code base is one step, but how can I know that the server running the service actually does run this codebase and not something else?

marksomnian 6y ago
This is a very difficult problem to solve. As far as I know (and am welcome to be corrected!), the current state of the art is remote attestation - Signal wrote a post about how they're doing it for their contact discovery service[0]. It's a very limited solution though. [0]: https://signal.org/blog/private-contact-discovery/ https://signal.org/blog/private-contact-discovery/
- cuu508 6y ago
  TL;DR they use Intel's SGX. Intel's 20GB data leak was on HN yesterday, with a hint that more is coming. Will be interesting to watch ;-)
- mLuby 6y ago
  IDK, trusting a remote system just seems bad. Either give me the data to process locally, or I'll give you some data you couldn't possibly understand and you do some stuff to it and return the results, having never known the meaning of the data. In Signal's case, they could just let you send messages that might never be received because the contact doesn't currently have the app. Also gives the receiver plausible deniability unless they respond. And could help get more people on Signal if they did the "Someone sent you a message, download the app now" trick.
coddle-hark 6y ago
Flipping the question around: How do YOU know that the server running the service is actually running the codebase and not something else? You might have been hacked. Someone at your hosting provider might be exfiltrating data. Intel might have a backdoor in their CPUs. It’s not practical for you to continuously verify all of these things, at the end of the day you have to trust that things do what they say they do until you have reason to believe otherwise. Of course, if you can’t verify what’s running in your server then neither can your users. All you can do is be as transparent as possible about how you process their data and hope that’s enough to earn their trust. As an aside, homomorphic encryption has been touted for a while now as a way for services to process encrypted user data without ever being able to decrypt it. Haven’t seen much use of it in the industry though, last I heard it was enormously impractical.
- mihaifm 6y ago
  You're actually implying that you can't trust any computer with the code you are telling it to execute. This is an exaggeration IMO. Sure you can have backdoors and hacks, but with what probability? On the other hand, users need to put their trust in humans, not computers.
  - xav0989 6y ago
    Ken Thompson wrote a paper about trust that was published in ACM in '84 entitled "Reflections on Trusting Trust". He makes the case for a trojan that could be built in such a way that it would infect a compiler, self-replicate into new versions of the compiler, and inject itself into specific binaries; all without appearing in the source code of the new versions of the compiler or the target binary. Arguably, the probability of such an attack is extremely low--slim to none might be an overstatement--but it's an interesting thought exercise.
    - IggleSniggle 6y ago
      I remember reading about this attack being implemented and executed in a university system decades ago. Perhaps apocryphal, but it didn’t seem that way.
      - SifJar 6y ago
        You're possibly thinking of this: https://www.quora.com/What-is-a-coders-worst-nightmare/answer/Mick-Stute https://www.quora.com/What-is-a-coders-worst-nightmare/answe...
        IggleSniggle 6y ago
        That's the one!
    - david_draco 6y ago
      All the godbolt users looking at their binaries from various compilers would quickly notice this, I think.
mirekrusin 6y ago
Good question, built in content hash on those lambda function would be great to have.
matsemann 6y ago
On the side of your question, but HIBP's API is designed in such a way that you don't have to give them your password to get notified if it has been hacked. You instead provide HIBP with parts of the hash, and then it answers if you maybe have been compromised. Take a look at requests with devtools on the page https://haveibeenpwned.com/Passwords https://haveibeenpwned.com/Passwords So here it's taken one step further, you don't have to trust that it's the same code running on HIBP, because you don't have to trust them at all for the service to work.
- faebi 6y ago
  But how many people will check in detail what‘s submitted? And again, how do you prevent that different users get different functionality?
  - prepend 6y ago
    Only a few need to check to see what’s submitted and serve as a canary for others. It’s possible some bad actor could do dumb stuff, but SSL helps in that at least the server would have a unique id. Personally the first time I used HIBP I checked the javascript that was running and read about what happened. It’s possible that someone could hack their server and replace the javascript with new bad versions. I don’t check anymore. But it’s unlikely, and a shared, common risk of using any site, I think. But the important thing with HIBP (and other sites) is that I never send them any sensitive information.
    - jhardy54 6y ago
      > Only a few need to check to see what’s submitted and serve as a canary for others. I couldn't agree more. Even if you have root on their server and do a full audit, there's nothing to stop them from changing the code the moment you log out.
  - mcherm 6y ago
    > But how many people will check in detail what‘s submitted? That's a valid criticism. It only takes one person finding clear evidence of problematic behavior to advertise that fact to the entire community. So long as a small fraction of people do actually check, the whole community will be fine. But if only a negligible number of people check it, then perhaps no one will be checking when it is abused. > And again, how do you prevent that different users get different functionality? Well, it depends on how the system is discriminating among users. To a great extent, that kind of abuse is prevented by anonymous users. You don't have to log into HIBP in order to use it. The system could still be discriminating by IP address, or even by various kinds of browser fingerprinting (including just collecting the advertising company cookies that so effectively eliminate anonymity for much of the web). Occasionally using TOR won't help against this -- a malicious operator could simply provide "clean" functionality to all TOR users. - - - - On the whole, things like, k-anonymous API design, making source code open, or having a small fraction of users checking for security issues all help a great deal and make it difficult for a malicious operator to abuse the system. But none of them are perfect. In the end it comes down to trust. Troy Hunt has EARNED that trust due to his openness, and in no small part by choosing to implement protections like these that he didn't need to implement, and I find that extremely persuasive. If you take steps to ensure that your CANNOT cheat your "customers", then that's pretty good evidence you aren't likely to be trying to cheat them.
londons_explore 6y ago
You make your webpage have all logic client-side, and use standardized serverside API's (for example google datastore, or amazon s3). Do not run any of your own application logic on the server - your javascript client should talk to the standardized datastore directly. Then your users can see all the logic on their side, and inspect all the data, and if they want they can host the client side stuff themselves and run their own server.
- bhandziuk 6y ago
  This doesn't seem sustainable either though. Isn't server-side code required to ensure the data you're getting from the client is legit. You can't do everything client side especially because not everything can be done in javascript.
  - londons_explore 6y ago
    > Isn't server-side code required to ensure the data you're getting from the client is legit. In this model, the client must validate the data before it puts it into the data store, and again when it gets it out of the data store (since the data could have been put there by a malicious/modified client)
jamesponddotco 6y ago
For my projects, I provide read-only SSH access when asked, but that probably does not work for everyone.
- pbhjpbhj 6y ago
  In theory you could maintain a separate system that the SSH accesses, changing the real system without revealing those changes.
  - jamesponddotco 6y ago
    True, and there is probably a way to prove that is not the case, but I have no idea what that way is right now. On the other hand, if you are bothering to email me to ask for SSH access to ensure what I say is true, you probably have the knowledge to detect if I am lying from inside the server.
renewiltord 6y ago
Why bother? Change your password to a randomly generated string then send over the old password. You've cut off the threat vector.

marksomnian 6y ago

This is a very difficult problem to solve. As far as I know (and am welcome to be corrected!), the current state of the art is remote attestation - Signal wrote a post about how they're doing it for their contact discovery service[0]. It's a very limited solution though. [0]: https://signal.org/blog/private-contact-discovery/ https://signal.org/blog/private-contact-discovery/

cuu508 6y ago
TL;DR they use Intel's SGX. Intel's 20GB data leak was on HN yesterday, with a hint that more is coming. Will be interesting to watch ;-)
mLuby 6y ago
IDK, trusting a remote system just seems bad. Either give me the data to process locally, or I'll give you some data you couldn't possibly understand and you do some stuff to it and return the results, having never known the meaning of the data. In Signal's case, they could just let you send messages that might never be received because the contact doesn't currently have the app. Also gives the receiver plausible deniability unless they respond. And could help get more people on Signal if they did the "Someone sent you a message, download the app now" trick.

coddle-hark 6y ago

Flipping the question around: How do YOU know that the server running the service is actually running the codebase and not something else? You might have been hacked. Someone at your hosting provider might be exfiltrating data. Intel might have a backdoor in their CPUs. It’s not practical for you to continuously verify all of these things, at the end of the day you have to trust that things do what they say they do until you have reason to believe otherwise. Of course, if you can’t verify what’s running in your server then neither can your users. All you can do is be as transparent as possible about how you process their data and hope that’s enough to earn their trust. As an aside, homomorphic encryption has been touted for a while now as a way for services to process encrypted user data without ever being able to decrypt it. Haven’t seen much use of it in the industry though, last I heard it was enormously impractical.

mihaifm 6y ago
You're actually implying that you can't trust any computer with the code you are telling it to execute. This is an exaggeration IMO. Sure you can have backdoors and hacks, but with what probability? On the other hand, users need to put their trust in humans, not computers.
- xav0989 6y ago
  Ken Thompson wrote a paper about trust that was published in ACM in '84 entitled "Reflections on Trusting Trust". He makes the case for a trojan that could be built in such a way that it would infect a compiler, self-replicate into new versions of the compiler, and inject itself into specific binaries; all without appearing in the source code of the new versions of the compiler or the target binary. Arguably, the probability of such an attack is extremely low--slim to none might be an overstatement--but it's an interesting thought exercise.
  - IggleSniggle 6y ago
    I remember reading about this attack being implemented and executed in a university system decades ago. Perhaps apocryphal, but it didn’t seem that way.
    - SifJar 6y ago
      You're possibly thinking of this: https://www.quora.com/What-is-a-coders-worst-nightmare/answer/Mick-Stute https://www.quora.com/What-is-a-coders-worst-nightmare/answe...
      - IggleSniggle 6y ago
        That's the one!
  - david_draco 6y ago
    All the godbolt users looking at their binaries from various compilers would quickly notice this, I think.

mirekrusin 6y ago

Good question, built in content hash on those lambda function would be great to have.

matsemann 6y ago

On the side of your question, but HIBP's API is designed in such a way that you don't have to give them your password to get notified if it has been hacked. You instead provide HIBP with parts of the hash, and then it answers if you maybe have been compromised. Take a look at requests with devtools on the page https://haveibeenpwned.com/Passwords https://haveibeenpwned.com/Passwords So here it's taken one step further, you don't have to trust that it's the same code running on HIBP, because you don't have to trust them at all for the service to work.

faebi 6y ago
But how many people will check in detail what‘s submitted? And again, how do you prevent that different users get different functionality?
- prepend 6y ago
  Only a few need to check to see what’s submitted and serve as a canary for others. It’s possible some bad actor could do dumb stuff, but SSL helps in that at least the server would have a unique id. Personally the first time I used HIBP I checked the javascript that was running and read about what happened. It’s possible that someone could hack their server and replace the javascript with new bad versions. I don’t check anymore. But it’s unlikely, and a shared, common risk of using any site, I think. But the important thing with HIBP (and other sites) is that I never send them any sensitive information.
  - jhardy54 6y ago
    > Only a few need to check to see what’s submitted and serve as a canary for others. I couldn't agree more. Even if you have root on their server and do a full audit, there's nothing to stop them from changing the code the moment you log out.
- mcherm 6y ago
  > But how many people will check in detail what‘s submitted? That's a valid criticism. It only takes one person finding clear evidence of problematic behavior to advertise that fact to the entire community. So long as a small fraction of people do actually check, the whole community will be fine. But if only a negligible number of people check it, then perhaps no one will be checking when it is abused. > And again, how do you prevent that different users get different functionality? Well, it depends on how the system is discriminating among users. To a great extent, that kind of abuse is prevented by anonymous users. You don't have to log into HIBP in order to use it. The system could still be discriminating by IP address, or even by various kinds of browser fingerprinting (including just collecting the advertising company cookies that so effectively eliminate anonymity for much of the web). Occasionally using TOR won't help against this -- a malicious operator could simply provide "clean" functionality to all TOR users. - - - - On the whole, things like, k-anonymous API design, making source code open, or having a small fraction of users checking for security issues all help a great deal and make it difficult for a malicious operator to abuse the system. But none of them are perfect. In the end it comes down to trust. Troy Hunt has EARNED that trust due to his openness, and in no small part by choosing to implement protections like these that he didn't need to implement, and I find that extremely persuasive. If you take steps to ensure that your CANNOT cheat your "customers", then that's pretty good evidence you aren't likely to be trying to cheat them.

londons_explore 6y ago

You make your webpage have all logic client-side, and use standardized serverside API's (for example google datastore, or amazon s3). Do not run any of your own application logic on the server - your javascript client should talk to the standardized datastore directly. Then your users can see all the logic on their side, and inspect all the data, and if they want they can host the client side stuff themselves and run their own server.

bhandziuk 6y ago
This doesn't seem sustainable either though. Isn't server-side code required to ensure the data you're getting from the client is legit. You can't do everything client side especially because not everything can be done in javascript.
- londons_explore 6y ago
  > Isn't server-side code required to ensure the data you're getting from the client is legit. In this model, the client must validate the data before it puts it into the data store, and again when it gets it out of the data store (since the data could have been put there by a malicious/modified client)

jamesponddotco 6y ago

For my projects, I provide read-only SSH access when asked, but that probably does not work for everyone.

pbhjpbhj 6y ago
In theory you could maintain a separate system that the SSH accesses, changing the real system without revealing those changes.
- jamesponddotco 6y ago
  True, and there is probably a way to prove that is not the case, but I have no idea what that way is right now. On the other hand, if you are bothering to email me to ask for SSH access to ensure what I say is true, you probably have the knowledge to detect if I am lying from inside the server.

renewiltord 6y ago

Why bother? Change your password to a randomly generated string then send over the old password. You've cut off the threat vector.