3 ms·

What an excellent counter-argument you pose. Well done. I guess I should have also put: Don't use a standard hashing algorithm without salt In addition, make

by axod 18y ago

What an excellent counter-argument you pose. Well done. I guess I should have also put: Don't use a standard hashing algorithm without salt In addition, make the salt secret - type it in when the program starts. Or as sho suggests transmit it from an out of jurisdiction site over a secure channel. To be honest I assumed those points would be obvious. We're on Hacker news, not Digg.

DenisM 18y ago
I didn't mean to insult you, my suggestion was sincere. Secure design is not something that can be relayed via small forum messages, the best I can do is plead for you to not implement those schemes but instead take more fundamental approach of either taking time to learn this subject in depth or leaving it to someone who has.
- axod 18y ago
  ok, thanks ;) . I have learnt a reasonable amount about the subject.. I'm still pretty confident that if all you possess is some hashes, with no knowledge of the hashing algorithm, salt etc, you won't be able to work out any of the IPs. If you think that assertion is false, please educate and explain why. But as you say, forums leave way too much room for misinterpretation in the finer details...
  - DenisM 18y ago
    Your design may be secure software design but it is not a secure system design. When a system consists of multiple components you need to take into account the security and the failure modes of each individual component as well interactions between those failures. The operator does not need to be tortured, he simply needs to be told to cough up the salt or get tried on terrorism charges in a secret court. How many people will you find willing to trade their freedom for your corporate profits? More importantly, you don't know whether they will or will not fold, regardless of what they tell you at the job interview. Next, if the operator is the only one to know the secret, what is to stop him from blackmailing your businesses? Or dying under the bus and taking the secret with him? You will have to introduce a backup person who also knows the secret and is also fallible in the same way, increasing your risk. The questions of loyalty and conflicting interests are collectively known as "the agency problem" - where agents act not consistent with the interests of their employers. Given agency problems soon enough you will come to a point where someone with big stake in the company (like a founder or CEO) has to know the secret to avoid possible conflict of interest. Of course such person will also have a lot to lose and will be even more likely to fold under simple pressure such as arrest of their assets. The idea with storing the salt out of jurisdiction also has weak spots in it: first, you can't use public CA because NSA likely has their own certs signed by all major CAs. You would need to roll your own SSL cert and deliver it physically. Next, there wouldn't be any machine-gun-toting people streaming into the datacenter - there will be a subpoena to the CEO and if he does not comply he will go to jail and lose his mansion. Is it possible to set data such that CEO can not fetch the data? Technically yes, but realistically CEO that does not control his secrets will likely feel uneasy about losing those secrets to an accident or an agency problem and so he will most likely try to assert control over said secret. At which point he becomes a point of attack. Now, how does remote site know not to dispense secrets to a anyone who asks? It would have to ask requester for an ID and that ID will be stored on the (now confiscated) machine. Then, who will administer the remote site? What are his legal and moral obligations? Can their government compel them to cough up the secrets on request of your government? Further, RAM can and does get examined. There are consumer-level tricks with canned air that allow to reboot the machine while keeping RAM content intact for the next booted OS to chew on. This was at the heart of the recent TPM security breach (for some implementations of TPM). Also when machines are being confiscated they are not unplugged, there are products that allow to transport machine without losing power. The system is complex and there will likely be bugs at the points where some components talk to some other components.
    - axod 18y ago
      Sure, these are all obvious attacks. But completely tangential to my point about the feasibility of the software. They're an 'operations/logistics' issue, not a software one.
      - DenisM 18y ago
        And this is what prompted my initial comment. End users don't care how good your software is - they care whtehr their informaiton is leaked or not. Designing this sort of secure software is akin to building the Maginot line - it may work as designed but will not fulfill the purpose.
sho 18y ago
Digg? No need to get insulting. Anyway, you're wrong about almost everything. You misunderstand what a hash function is. It is not an encryption algorithm. Both the hash and the salt are supposed to be known to the attacker. And by the way, salts are supposed to be unique per record. An ideal hash function is supposed to be an irreversible, unfakeable process regardless of the attacker's knowledge of all components going in. That knowledge is assumed! My whole point is that while this is a very useful tool, it doesn't work for data sets small enough to brute force. This is elementary. Your response, cobbling together some kind of poor man's encryption by padding out the plaintext before hashing with a secret salt, is ridiculous. If you have a secret key then just ENCRYPT! Geeze man, GP was a bit rude, but if you're this stubborn about something you're obviously wrong about, maybe you should keep far away from security ..
- axod 18y ago
  "but if you're this stubborn about something you're obviously wrong about" Look, we just established on a separate thread that the system I described was secure (Barring memory dumps or torture of the operator). That's not being wrong. As you suggested, yes you could go all out and encrypt, but that would have practical limitations which would make it less desirable. Perhaps I was wrong to start a discussion about security at all.
- axod 18y ago
  Hey :) I know what a hash function is. I also know the trade offs between using a hash and full blown encryption. "salts are supposed to be public. Furthermore, they're supposed to be unique per record." There are no rules you know :/ For something like this, where you're just trying to make a log of IPs unreadable by the government/hackers/whatever, I'd still say a hash is the best bet. Your point about may as well encrypt is a valid one, but may result in higher complexity/cpu usage/mem usage/storage, for not much added security. It depends just how secure you want to be. I think we've flogged this one to death though :)
  - sho 18y ago
    Well, true, there are no hard and fast rules. Whatever works. Some things work better than others, though! The hypothetical system we're arguing about is kind of ridiculous anyway. No-one is going to go to these lengths to protect just one tiny aspect of their operation, and if they did, it would be dwarfed by everything else. And this hypothetical secret-keeping operator is just going to tell the cops the secret within 5 minutes anyway. You can't "defeat" a country's own government, on their turf, like this. And let's not even think about the time we've wasted arguing about storing the IPs when the real solution would seem to be to just not store the damn search string .. Lol, this horse is dead all right.

by axod 18y ago

DenisM 18y ago

I didn't mean to insult you, my suggestion was sincere. Secure design is not something that can be relayed via small forum messages, the best I can do is plead for you to not implement those schemes but instead take more fundamental approach of either taking time to learn this subject in depth or leaving it to someone who has.

axod 18y ago
ok, thanks ;) . I have learnt a reasonable amount about the subject.. I'm still pretty confident that if all you possess is some hashes, with no knowledge of the hashing algorithm, salt etc, you won't be able to work out any of the IPs. If you think that assertion is false, please educate and explain why. But as you say, forums leave way too much room for misinterpretation in the finer details...
- DenisM 18y ago
  Your design may be secure software design but it is not a secure system design. When a system consists of multiple components you need to take into account the security and the failure modes of each individual component as well interactions between those failures. The operator does not need to be tortured, he simply needs to be told to cough up the salt or get tried on terrorism charges in a secret court. How many people will you find willing to trade their freedom for your corporate profits? More importantly, you don't know whether they will or will not fold, regardless of what they tell you at the job interview. Next, if the operator is the only one to know the secret, what is to stop him from blackmailing your businesses? Or dying under the bus and taking the secret with him? You will have to introduce a backup person who also knows the secret and is also fallible in the same way, increasing your risk. The questions of loyalty and conflicting interests are collectively known as "the agency problem" - where agents act not consistent with the interests of their employers. Given agency problems soon enough you will come to a point where someone with big stake in the company (like a founder or CEO) has to know the secret to avoid possible conflict of interest. Of course such person will also have a lot to lose and will be even more likely to fold under simple pressure such as arrest of their assets. The idea with storing the salt out of jurisdiction also has weak spots in it: first, you can't use public CA because NSA likely has their own certs signed by all major CAs. You would need to roll your own SSL cert and deliver it physically. Next, there wouldn't be any machine-gun-toting people streaming into the datacenter - there will be a subpoena to the CEO and if he does not comply he will go to jail and lose his mansion. Is it possible to set data such that CEO can not fetch the data? Technically yes, but realistically CEO that does not control his secrets will likely feel uneasy about losing those secrets to an accident or an agency problem and so he will most likely try to assert control over said secret. At which point he becomes a point of attack. Now, how does remote site know not to dispense secrets to a anyone who asks? It would have to ask requester for an ID and that ID will be stored on the (now confiscated) machine. Then, who will administer the remote site? What are his legal and moral obligations? Can their government compel them to cough up the secrets on request of your government? Further, RAM can and does get examined. There are consumer-level tricks with canned air that allow to reboot the machine while keeping RAM content intact for the next booted OS to chew on. This was at the heart of the recent TPM security breach (for some implementations of TPM). Also when machines are being confiscated they are not unplugged, there are products that allow to transport machine without losing power. The system is complex and there will likely be bugs at the points where some components talk to some other components.
  - axod 18y ago
    Sure, these are all obvious attacks. But completely tangential to my point about the feasibility of the software. They're an 'operations/logistics' issue, not a software one.
    - DenisM 18y ago
      And this is what prompted my initial comment. End users don't care how good your software is - they care whtehr their informaiton is leaked or not. Designing this sort of secure software is akin to building the Maginot line - it may work as designed but will not fulfill the purpose.

sho 18y ago

Digg? No need to get insulting. Anyway, you're wrong about almost everything. You misunderstand what a hash function is. It is not an encryption algorithm. Both the hash and the salt are supposed to be known to the attacker. And by the way, salts are supposed to be unique per record. An ideal hash function is supposed to be an irreversible, unfakeable process regardless of the attacker's knowledge of all components going in. That knowledge is assumed! My whole point is that while this is a very useful tool, it doesn't work for data sets small enough to brute force. This is elementary. Your response, cobbling together some kind of poor man's encryption by padding out the plaintext before hashing with a secret salt, is ridiculous. If you have a secret key then just ENCRYPT! Geeze man, GP was a bit rude, but if you're this stubborn about something you're obviously wrong about, maybe you should keep far away from security ..

axod 18y ago
"but if you're this stubborn about something you're obviously wrong about" Look, we just established on a separate thread that the system I described was secure (Barring memory dumps or torture of the operator). That's not being wrong. As you suggested, yes you could go all out and encrypt, but that would have practical limitations which would make it less desirable. Perhaps I was wrong to start a discussion about security at all.
axod 18y ago
Hey :) I know what a hash function is. I also know the trade offs between using a hash and full blown encryption. "salts are supposed to be public. Furthermore, they're supposed to be unique per record." There are no rules you know :/ For something like this, where you're just trying to make a log of IPs unreadable by the government/hackers/whatever, I'd still say a hash is the best bet. Your point about may as well encrypt is a valid one, but may result in higher complexity/cpu usage/mem usage/storage, for not much added security. It depends just how secure you want to be. I think we've flogged this one to death though :)
- sho 18y ago
  Well, true, there are no hard and fast rules. Whatever works. Some things work better than others, though! The hypothetical system we're arguing about is kind of ridiculous anyway. No-one is going to go to these lengths to protect just one tiny aspect of their operation, and if they did, it would be dwarfed by everything else. And this hypothetical secret-keeping operator is just going to tell the cops the secret within 5 minutes anyway. You can't "defeat" a country's own government, on their turf, like this. And let's not even think about the time we've wasted arguing about storing the IPs when the real solution would seem to be to just not store the damn search string .. Lol, this horse is dead all right.