3 ms·
How is it not differentiable in its logarithm? The function is just f(x) = sin^2(A * B^x), for some constants A and B (A encodes the data while B just determin
by comex 8y ago
How is it not differentiable in its logarithm? The function is just f(x) = sin^2(A * B^x), for some constants A and B (A encodes the data while B just determines the precision). The function itself is infinitely differentiable, so its logarithm will be infinitely differentiable everywhere except where f(x) = 0. However, it's not zero at the points that matter, and in any case we can avoid the issue by just adding a constant.
- AstralStorm 8y agoNot the function itself, the log ratio of fit probabilities of any given pair of these. That is differentiable at most the Lebesgue sense and the likelihood requires something stronger. Specifically it has to be smooth everywhere to have KL divergence well defined. Adding a constant will give you a logarithm that breaks at zero still in log probability ratio. So, both BIC and AIC are ill defined for this family of functions... Part of the reason the measure returns worthless garbage. This also happens with fits based on neural nets with tan or clipped activations. (Because sum of activations is nonsmooth as is fit probability.) But not with RBM or GMM or exponential neurons. These produce Gaussian or Pareto fit probabilities. (Polynomial probably also fail because they're not smooth functions but both measures could be corrected for nonsmoothness of likelilihood in this case.) Sum of sinc should work too as you get Wishart fit probabilities. This funny chaotic function? I have no idea how distributed the answers are and whether the distribution is continuous.