5 ms·
This article brazenly glosses over HTTPS, which is designed to tackle the very issues that are raised (privacy and authenticity). APIs do not need to sign every
by speedplane 7y ago
This article brazenly glosses over HTTPS, which is designed to tackle the very issues that are raised (privacy and authenticity). APIs do not need to sign every response, they just need to properly use HTTPS.
- jayd16 7y agoThe author seems to be aware that https has this function as they seem to state using it ensures the response they see is valid. However, they don't seem to understand why that is or that the raw response would remain verifiable to others.
- kelnos 7y agoThis is addressed in the article specifically. They're not talking about the person making the API request needing a signature to verify its authenticity. It's about if someone else sends you an API response that they retrieved, you should be able to verify it wasn't tampered with.
- speedplane 7y ago> They're not talking about the person making the API request needing a signature to verify its authenticity. It's about if someone else sends you an API response that they retrieved, you should be able to verify it wasn't tampered with. If a system retrieves data, it should do so with HTTPS, and then it's authenticity can be guaranteed. If it does not use HTTPS (or some similar protocol), then it's authenticity cannot be guaranteed and no amount of signing or hashing can fix that.
- notduncansmith 7y agoMost access to HTTPS is transparent, i.e. signatures are not that easy to extract and independently verify (not to mention all the issues with CA infrastructure)... Basically, while maybe you could technically accomplish it with HTTPS, there are huge usability gains to be had in exchange for the inefficiency of duplicating some lower-level functionality at a higher level.
- speedplane 7y ago> while maybe you could technically accomplish it with HTTPS, there are huge usability gains to be had in exchange for the inefficiency of duplicating some lower-level functionality If you are not using HTTPS, you cannot just build your own "higher level" version of secure and authenticated communication. HTTPS works because trusted certificates are pre-installed on each of the machines before they start communicating. Without those pre-existing trusted certificates, no amount of encryption, signing, or hashing can guarantee secure and authenticated communication.
- deleted 7y ago[deleted]
- notduncansmith 7y agoI agree that from a practical perspective, the best way to secure one’s communications with a web server is to enable it with HTTPS. I also think though that it’s possible, for any number of reasons, that one might want signed API responses independently of that (e.g. OTA software updates). Once you get into cryptographic identity though, you’re back to CA/PKI all over again...
- speedplane 7y ago> it's s possible, for any number of reasons, that one might want signed API responses independently At it's core, HTTPS works because there is a distributed system of trust. The principle is that you don't need to directly trust the server that you're communicating with, but you trust another computer that has vouched for the one you are communicating with. If you roll your own security in any way, at some point you need to send an encryption or decryption key. Even "unhackable" encryption algorithms are worthless if your adversary knows the key. Thus, you need a way to communicate that initial key information securely. Currently, the only way to do that is by (1) manually entering the key on each server, (2) using near sci-fi technology like quantum communication, or (3) by having the keys pre-installed on every machine by the hardware manufacturer or OS provider. The option that has worked, HTTPS, uses option 3. Maybe you can roll your own encrypted communication system, but you'll either have to rely on manually entering the keys (which in turn rely on other insecure methods like email and phone), or trust the computer's manufacturer to pre-install the keys on the system your using. There really is no way to "roll your own" system securely without these steps.
- codegladiator 7y agoOP is talking about system 1 getting a response generated by system 3 VIA system 2 (potentially trusted man in the middle). And they are already assuming that system 1 and 2 talk using https and system 2 and 3 talk using https.
- speedplane 7y ago> OP is talking about system 1 getting a response generated by system 3 VIA system 2 (potentially trusted man in the middle). So they are effectively saying that they do not trust system 2. If that's the case, a signature from system 2 would not help because the signature itself could not be trusted. They would need the signature from the original source, system 3.
- codegladiator 7y ago> a signature from system 2 That's why they are talking about signature from system 3 along with the payload. So even if they trust system 2, they can be sure system 2 has not tampered with the message generated by system 3.
- speedplane 7y ago> That's why they are talking about signature from system 3 along with the payload. So even if they trust system 2, they can be sure system 2 has not tampered with the message generated by system 3. No that wouldn't work by itself. System 2 could tamper the data, create their own signature, and send it to system 1. The only way it could work is if System 1 got the signature directly from System 3, or at least got System 3's public signing certificate. Either way, System 1 needs to communicate directly with System 3 to ensure the data it's getting from System 2 is authentic.
- codegladiator 7y agoI think you are right. At some point system 1 will need to talk to system 3 to validate the message (unless system 1 and 3 have an already shared secret or public key)
- kelnos 7y agoI'm not really sure how to explain this any better, but let's try: Our cast of characters: Service A, Person 1, and Person 2. Person 1 makes an API request to Service A. It's done over HTTPS, and Person 1 trusts that the response received is genuine, because it came over HTTPS, and certs were verified and so forth. Person 1 sends the text of the API response to Person 2. Let's say Person 2 is a journalist, and Person 1 is a potential source. Person 2 doesn't know Person 1, and so logically doesn't blindly trust them. But Person 2 wants to report on some factual thing about the API response. How can Person 2 be sure that Person 1 hasn't modified the API response before Person 2 receives it? If the API response is signed somehow, Person 2 could go retrieve Service A's public key (directly from Service A) and independently validate that the API response was actually received from Service A, and that it hasn't been tampered with. I'm not sure I agree with the original author that this is a critical problem that needs to be solved, but that's the gist of it.
- speedplane 7y ago> How can Person 2 be sure that Person 1 hasn't modified the API response before Person 2 receives it? ... If the API response is signed somehow, Person 2 could go retrieve Service A's public key (directly from Service A) and independently validate that the API response was actually received The problem here is that Person 1 cannot be trusted, so they can't be the one providing the signature. Even if Person 1 signs, hashes, or encrypts the message when sending to Person 2, all it can prove is that it came from Person 1, not that it came from Service A. If Person 2 wants to be sure that the message it got from Person 1 actually came from Service A, it will need the signature provided by Service A, not Person 1. To get the valid signature, it will need to contact Service A. This is a long-winded way to say that if you want to trust what you got, you've got to go to the original source. HTTPS provides tools to do exactly this, and it's special because every time you buy a computer or server, trusted certificates are pre-installed on the machine. Without that initial level of trust, you can't trust anything you receive from anywhere.
- jayd16 7y agoThey can share the exact same encrypted bytes they received. Others can verify those bytes with the cert as normal. Or do I have it wrong, and the traffic is only signed with the user signature after the handshake?
- edent 7y agoYou have it wrong. I also thought that was the answer. See these responses to my question on StackExchange https://security.stackexchange.com/questions/193559/saving-and-verifying-an-https-request https://security.stackexchange.com/questions/193559/saving-a...
- jayd16 7y agoLooks like the message is encrypted with a key derived by both parties but this produces a symmetric encryption. The client then has the ability to spoof encrypted server communications.
- tsimionescu 7y agoSSL data is encrypted with a symmetric, per-session key. Only the initial handshake is done through asymmetric encryption, and only the initial handshake uses the global public/private key pair.
- deleted 7y ago[deleted]
- deleted 7y ago[deleted]
- OrderlyTiamat 7y agoI appreciate the point the author is making, but why is this a problem of the api provider? And why would api providers be incentivised to implement the solutions to this problem?
- kelnos 7y agoI'm not sure they would be. They are the best party to sign/verify that responses are genuine, though.
- sigsergv 7y agoUnfortunately in some (most?) cases you don't control list of trusted authorities on the endpoint.
- bigiain 7y agoThat's not how I read the article. This bit from right near the beginning explains why he's interested in more than what https provides: If I call https://api.example.com/?id=123 https://api.example.com/?id=123 myself, then I can be reasonably sure that the data has come from the server unaltered. But what if someone emails me a JSON file? Or someone re-hosts an XML dump? Or a hacker claims to have uncovered proof of...? As data-journalism gathers pace, it's vital to be able to see whether data is authentic.
- zorked 7y agoHow do I know your comment is legit if you didn't sign it? What is special about APIs?
- nofunsir 7y agoSign all the things! But for real. I had an idea a while back: with all the photo/audio/video manipulation and ‘deep fakes’, there should be a certification available that accompanies (very expensive) cameras/recorders with protected firmware that signs each and every raw frame/recording, so the original is always available upon request from any type of inquiry or investigation.
- byteshock 7y agoSounds like an invasion of privacy. Your personal pictures and videos shouldn’t have a unique digital signature so the police or a bad actor can identify which device it was taken on. This seems like a good idea at first but what about whistleblowers or informants?
- AndrewDucker 7y agoWouldn't have to tie it back to you. It would be enough to say "Canon affirms this photo was taken directly on a model X". Which would then tell you that this was the unedited photo and could be trusted.
- 7y ago
- benmmurphy 7y agoPart of HTTPs negotiates a bunch of symmetric keys and these are used to encrypt and ensure the integrity of the data. Because the keys are symmetric it is possible for either party to fake a transcript of the conversation after the point that the symmetric keys are negotiated. So you cannot use HTTPs to prove to a third party a response or a request when using client certificates is genuine.
- speedplane 7y ago> So you cannot use HTTPs to prove to a third party a response or a request when using client certificates is genuine. HTTPS is not sufficient for such third-party exchange, but they are necessary. If you (say "System 1") ask for a twitter feed from an intermediary (say "System 2"), which in turn scrapes the data from twitter itself (say "System 3"), then HTTPS can indeed help. You can't rely on any hash or signature produced by System 2, because they could modify the data and make up a new hash. Instead, you would need to rely on the hash from System 3, to confirm that what you received from System 2 is authentic. Only tools like HTTPS, with their pre-installed security certificates, can do that.
- raxxorrax 7y agoI think there could be a case made based on the fact that the HTPPS connection isn't necessarily established by the service you want to connect to. Especially in the cloud world where TLS can be terminated on a load balancer or similar network service instead of the service you wanted to address. This assumes threats between these two services, similar to what the author describes as manipulation by intermediaries. In this case the author criticizes manipulation of the content by third parties. I hope this is not another argument for a digital notary and believe that there are better technical solutions to that problem. Digital ledgers seem to be a niche some people try to establish. In the end HTTPS is just transport encryption, not content encryption or content signing. From an architecture point of view I think it is even advantageous if not every services handles HTTPS itself.
- rumanator 7y ago> APIs do not need to sign every response, they just need to properly use HTTPS. That doesn't make sense. The problem described in the article was that the request documents aren't signed, thus there is no way to verify if a response document was actually outputted by a web service. HTTPS handles transport, but has no responsibility regarding request documents. If the problem we need to solve consists of ensuring if a given JSON was actually outputted as a response to a HTTP request to a service (say, if a Twitter message was actually sent by user X on day Y) then how would HTTPS help?
- speedplane 7y ago>> APIs do not need to sign every response, they just need to properly use HTTPS. > That doesn't make sense. The problem described in the article was that the request documents aren't signed, thus there is no way to verify if a response document was actually outputted by a web service. Suppose that you ("System 1") is getting data from an intermediary ("System 2") which in turn is getting data from Twitter itself ("System 3"). Even if the intermediary System 2 signs the data, all that proves is that you got it from System 2. It does not prove that it was not tampered with when System 2 got that data from twitter directly, System 3. It would be easy for System 2 to tamper with the data, and come up with it's own signature. If you, as System 1, want to confirm that the message you got from the intermediary (System 2) is authentically from twitter itself (System 3), you would need to get the signature from twitter directly. You would need to ask for twitter's signature (System 3's signature), not the intermediary's. So system 1 will need to contact System 3 in order to validate the information it got from System 2. There's no way around this.
- tsimionescu 7y agoDepend on your use case, HTTPS may not give you the right guarantees. For example, if you are trying to check an online license (where the client running the current application is essentially a potential adversary), HTTPS gives you no guarantees that the license server you think you are talking to is the real license server. You could pin the certificate to work around this, but then you have to reinvent revocation and renewal of the certificate for your own application. A longer lived, application specific signature may be a better option. In other cases, HTTPS's end-to-end encryption may not be completely desirable - for example, apt does not use HTTPS as it would interfere with simple caching proxies, and it offers no extra assurances than a verified signature does.
- speedplane 7y ago> HTTPS gives you no guarantees that the license server you think you are talking to is the real license server What? This is the entire purpose of HTTPS! To do this securely: 1 - you create your server; 2 - you tell a certificate authority (like Symantec, Digicert, or Let's Encrypt) how to find your server, and they do the diligence to make sure you're not trying to redirect another person's server; 3 - they provide you with a private encryption key and expose the public decryption key; 4 - you save the encryption key on your server. Now, anyone who receives any information from your server can be sure that it came from you because (a) your server is the only one with the encryption key and (b) we trust Symantec, Digitcert, etc to distribute the public decryption key.