3 ms·

I know that its more the capabilities angle that the US Government is mainly interested in here, rather than copyright infringement... But just curious, could p

I know that its more the capabilities angle that the US Government is mainly interested in here, rather than copyright infringement... But just curious, could packages like google's fully-homomorphic-encryption or the difficult to build Pyfhel be used to mitigate government intrusion into model training / datasets? I know there is some work being done to 'extract' training data from models (although given the compression, not entirely sure you could really extract everything but curious if anyone is seriously working on training models on purely encrypted data

PeterisP 3y ago
FHE does still add a huge overhead (measured in orders of magnitude, not percentages), so it would not be applicable for large models that are comparable with state of art models. You can apply FHE to neural networks in general (e.g. https://developers.googleblog.com/2023/08/expanding-our-fully-homomorphic-encryption-offering.html https://developers.googleblog.com/2023/08/expanding-our-full... lists it as one of potential use cases), but the unsaid assumption is that they are relatively small NNs, and that's in an era where we know that simply using a larger model brings meaningful improvements to the outcome, and the main limiting factor for people training their own models is the cost of compute.