4 ms·

They could just be normalizing every data point to be between 0 and 1 by dividing by the range. That's a homomorphic encryption.. it passes your weird assumptio

by jamez1 10y ago

They could just be normalizing every data point to be between 0 and 1 by dividing by the range. That's a homomorphic encryption.. it passes your weird assumptions. I don't know why you're harping on about sound encryption, the point of this is to keep the statistical information intact in the cipher, without giving away the underlying market data.

Houshalter 10y ago
It's very weird to call data normalization "encryption". This is just a standard procedure done on most datasets. Encryption implies they've gone to extra processing to make sure you can't figure out what the variables mean. I think it's either an abuse of the word 'encryption'. That, or they really have done something weird to this dataset. Which will probably make it useless for statistical algorithms. Even normalization destroys a lot of useful information.
- jamez1 10y ago
  They only need to go as far as to obfuscate the market data this was derived from, so they don't have to pay exchange licencing fees. It doesn't have to have an exponential time complexity on decryption to qualify as 'encryption'. Multiplying by 2 could be considered homomorphic encryption. You might think encryption means something else and that it's an abuse of the word but unlike the spy novella that you derive this impression from, these guys actually are ex-spies.
  - nathan_f77 10y ago
    > They only need to go as far as to obfuscate the market data this was derived from, so they don't have to pay exchange licencing fees. Thanks for explaining this, I was struggling to figure out the difference between this and https://www.quantopian.com/ https://www.quantopian.com/ That's actually pretty clever.
dooglius 10y ago
What I'm talking about is indistinguishability [0], [1] which shows up in definitions of homomorphic encryption (e.g. [2], [3]). If the data is only normalized, or even encrypted in an order-preserving way, it seems possible to figure out information about the underlying data (e.g. if the target is whether a symbol moves up or down, and if you can figure out what even one of the features refer to, there's enough information to turn your model on the data into predictions you can just trade on). [0] https://en.wikipedia.org/wiki/Computational_indistinguishability https://en.wikipedia.org/wiki/Computational_indistinguishabi... [1] https://en.wikipedia.org/wiki/Ciphertext_indistinguishability https://en.wikipedia.org/wiki/Ciphertext_indistinguishabilit... [2] http://cs.au.dk/~stm/local-cache/gentry-thesis.pdf http://cs.au.dk/~stm/local-cache/gentry-thesis.pdf [3] https://arxiv.org/ftp/arxiv/papers/1305/1305.5886.pdf https://arxiv.org/ftp/arxiv/papers/1305/1305.5886.pdf

by jamez1 10y ago

Houshalter 10y ago

It's very weird to call data normalization "encryption". This is just a standard procedure done on most datasets. Encryption implies they've gone to extra processing to make sure you can't figure out what the variables mean. I think it's either an abuse of the word 'encryption'. That, or they really have done something weird to this dataset. Which will probably make it useless for statistical algorithms. Even normalization destroys a lot of useful information.

jamez1 10y ago
They only need to go as far as to obfuscate the market data this was derived from, so they don't have to pay exchange licencing fees. It doesn't have to have an exponential time complexity on decryption to qualify as 'encryption'. Multiplying by 2 could be considered homomorphic encryption. You might think encryption means something else and that it's an abuse of the word but unlike the spy novella that you derive this impression from, these guys actually are ex-spies.
- nathan_f77 10y ago
  > They only need to go as far as to obfuscate the market data this was derived from, so they don't have to pay exchange licencing fees. Thanks for explaining this, I was struggling to figure out the difference between this and https://www.quantopian.com/ https://www.quantopian.com/ That's actually pretty clever.

dooglius 10y ago

What I'm talking about is indistinguishability [0], [1] which shows up in definitions of homomorphic encryption (e.g. [2], [3]). If the data is only normalized, or even encrypted in an order-preserving way, it seems possible to figure out information about the underlying data (e.g. if the target is whether a symbol moves up or down, and if you can figure out what even one of the features refer to, there's enough information to turn your model on the data into predictions you can just trade on). [0] https://en.wikipedia.org/wiki/Computational_indistinguishability https://en.wikipedia.org/wiki/Computational_indistinguishabi... [1] https://en.wikipedia.org/wiki/Ciphertext_indistinguishability https://en.wikipedia.org/wiki/Ciphertext_indistinguishabilit... [2] http://cs.au.dk/~stm/local-cache/gentry-thesis.pdf http://cs.au.dk/~stm/local-cache/gentry-thesis.pdf [3] https://arxiv.org/ftp/arxiv/papers/1305/1305.5886.pdf https://arxiv.org/ftp/arxiv/papers/1305/1305.5886.pdf