zakeria

3 ms·

1. 1.▲
   by zakeria 1y ago

   Thanks for the comment - uGMM neurons are not just "RBFs with log outputs". Each neuron is a mixture of Gaussians with trainable means, variances, and mixture weights, encoding uncertainty and multimodality that propagates through
2. 2.▲
   by zakeria 1y ago

   Thanks - That's correct, the Gaussian mixture parameters (mu, sigma, pi) are learned as functions of the input from the previous layer. So it’s still a feedforward net: the activations from layer x determine the mixture parameters for
3. 3.▲
   by zakeria 1y ago

   That’s a fair question. You’re right that on paper a uGMM neuron looks like it “costs” ~3× an MLP weight. But there are levers to balance that. For example, the paper discusses parameter tying, where the Gaussian component means are tied di
4. 4.▲
   by zakeria 1y ago

   Thanks - good question, in theory, the uGMM layer could complement CNNs in different ways - for example, one could imagine (as you mentioned): using standard convolutional layers for feature extraction, then replacing the final dense layers
5. 5.▲
   by zakeria 1y ago

   Thanks for the comment. Just to clarify, the uGMM-NN isn't simply "Gaussian sampling along the parameters of nodes." Each neuron is a univariate Gaussian mixture with learnable mean, variance, and mixture weights. This gives
6. 6.▲
   UGMM-NN: Univariate Gaussian Mixture Model Neural Network(arxiv.org)

   31 points by zakeria 1y ago|12 comments
7. 7.▲
   by zakeria 1y ago

   uGMM-NN is a novel neural architecture that embeds probabilistic reasoning directly into the computational units of deep networks. Unlike traditional neurons, which apply weighted sums followed by fixed nonlinearities, each uGMM-NN node par
8. 8.▲
   Show HN: UGMM-NN – A FF neural network using univariate Gaussian mixture neurons(github.com)

   2 points by zakeria 1y ago|0 comments
9. 9.▲
   by zakeria 7y ago

   Fast forward a few years, now with the demand of learning-based software, there is no doubt that mathematics is indeed a requirement for data-driven software. Although you may not need to know the math to build such software, given the tool