3 ms·

I was told by an engineer from Leapyear Technologies (https://leapyear.io/ https://leapyear.io/) that this library was mostly primitive functionalities that are

I was told by an engineer from Leapyear Technologies (https://leapyear.io/ https://leapyear.io/) that this library was mostly primitive functionalities that are behind the current mainstream practices. Disclaimer: Myself not an expert in the field.

bo1024 6y ago
I think "behind" is not the right way to put it. It only has the basic functionalities, but these are crucial and the implementation looks state-of-the-art to me (at a glance). It doesn't have the fancier machine-learning algorithms, true.
drfuchs 6y ago
Leapyear has a financial interest in having you believe that. Are their algorithms open-source? If not, can you buy a copy, or is it closed-source, SaaS-only?
ThePhysicist 6y ago
Applying DP to a simple computation like an average or median isn't that hard, what's more tricky is to ensure reasonable privacy guarantees when allowing unlimited interactive queries or large-scale sample generation from high-dimensional data: You can apply DP to individual datapoints or attributes, but the amount of noise that you need to add to reach reasonable privacy guarantees is then quite high. Hence it makes more sense to add noise to the result of a computation, as the sensitivity of many practically relevant computations to individual datapoint values will often be small, hence the required amount of noise to mask the contribution of each individual datapoint will also be low. The problem is just that often a single datapoint can contribute to the result of many (sometimes nearly infinitely many) computations, and every DP computation result you return to a user (or adversary) will reduce you privacy budget. There are some approaches to remedy this problem, like adding "sticky noise" or remembering queries to ensure no averaging of noise is possible, but all of them have their drawbacks. Therefore we still see quite limited use of DP in interactive data analysis or machine learning, because it is quite hard to strictly ensure reasonable privacy guarantees in those cases. Would be interesting to know if LeapYear has come up with something better, but they don't seem to have any source code or datasets available for public scrutiny.

bo1024 6y ago

I think "behind" is not the right way to put it. It only has the basic functionalities, but these are crucial and the implementation looks state-of-the-art to me (at a glance). It doesn't have the fancier machine-learning algorithms, true.

drfuchs 6y ago

Leapyear has a financial interest in having you believe that. Are their algorithms open-source? If not, can you buy a copy, or is it closed-source, SaaS-only?

ThePhysicist 6y ago

Applying DP to a simple computation like an average or median isn't that hard, what's more tricky is to ensure reasonable privacy guarantees when allowing unlimited interactive queries or large-scale sample generation from high-dimensional data: You can apply DP to individual datapoints or attributes, but the amount of noise that you need to add to reach reasonable privacy guarantees is then quite high. Hence it makes more sense to add noise to the result of a computation, as the sensitivity of many practically relevant computations to individual datapoint values will often be small, hence the required amount of noise to mask the contribution of each individual datapoint will also be low. The problem is just that often a single datapoint can contribute to the result of many (sometimes nearly infinitely many) computations, and every DP computation result you return to a user (or adversary) will reduce you privacy budget. There are some approaches to remedy this problem, like adding "sticky noise" or remembering queries to ensure no averaging of noise is possible, but all of them have their drawbacks. Therefore we still see quite limited use of DP in interactive data analysis or machine learning, because it is quite hard to strictly ensure reasonable privacy guarantees in those cases. Would be interesting to know if LeapYear has come up with something better, but they don't seem to have any source code or datasets available for public scrutiny.