3 ms·

Thanks for the submission. I keep an eye out for connections to potentially cheaper and simpler methods. The new comparison reminds me of this old paper which c

Thanks for the submission. I keep an eye out for connections to potentially cheaper and simpler methods. The new comparison reminds me of this old paper which compared them to Gaussian networks: https://arxiv.org/abs/1711.00165 https://arxiv.org/abs/1711.00165 For causality, I'm also keeping an eye out for ways to tie DNN research back into decision trees, esp probabilistic and nested. Or fuzzy logic. Here's an example I saw moving probabilistic methods into decision trees: http://www.gatsby.ucl.ac.uk/~balaji/balaji-phd-thesis.pdf http://www.gatsby.ucl.ac.uk/~balaji/balaji-phd-thesis.pdf Many of these sub-fields develop separately in their own ways. Gotta wonder what recent innovations in one could be ported to another.

yazzku 3y ago
This seems like the key research to me if we want any shot at preventing the technology from being locked away behind big corp API walls interfacing giant data centers. Anything that removes the bloat and mysticism from the models so they can be scaled down and run on the little guy's computer is orders of magnitude more progress in my opinion than, e.g., increasing the token window by some epsilon.
westurner 3y ago
Does a given model converge after Gaussian blurring? What does it do in the presence of noise, given the curse of dimensionality? OpenCog integrates PLN and MOSES (~2005). "Interpretable Model-Based Hierarchical RL Using Inductive Logic Programming" (2021) https://news.ycombinator.com/item?id=37463686 https://news.ycombinator.com/item?id=37463686 : > https://en.wikipedia.org/wiki/Probabilistic_logic_network https://en.wikipedia.org/wiki/Probabilistic_logic_network : >> The basic goal of PLN is to provide reasonably accurate probabilistic inference in a way that is compatible with both term logic and predicate logic, and scales up to operate in real time on large dynamic knowledge bases Asmoses updates an in-RAM (*) online hypergraph with the graph relations it learns. CuPy wraps CuDNN. Re: quantum logic and quantum causal inference: https://news.ycombinator.com/item?id=38721246 https://news.ycombinator.com/item?id=38721246 From https://news.ycombinator.com/item?id=39255303 https://news.ycombinator.com/item?id=39255303 : > Is Quantum Logic the correct propositional logic? Is Quantum Logic a sufficient logic for all things? A quantum ML task: Find all local and nonlocal state linkages within the presented observations And then also do universal function approximation
- westurner 3y ago
  But also biological neurological systems; Coping strategies: https://en.wikipedia.org/wiki/Coping https://en.wikipedia.org/wiki/Coping Defense Mechanisms > Vaillant's categorization > Level 4: mature: https://en.wikipedia.org/wiki/Defence_mechanism#Level_4:_mature https://en.wikipedia.org/wiki/Defence_mechanism#Level_4:_mat... "From Comfort Zone to Performance Management" suggests that the Carnall coping cycle coincides with the TPR curve (Transforming, Performing, Reforming, adjourning); that coping with change in systems is linked with performance. And Consensus; social with nonlinear feedback and technological. What are the systems thinking advantages in such fields of study? Systems theory > See also > Glossary,: https://en.wikipedia.org/wiki/Systems_theory#See_also https://en.wikipedia.org/wiki/Systems_theory#See_also
  - westurner 3y ago
    Wrong thread, my mistake; this comment was for this thread: https://news.ycombinator.com/context?id=39504104 https://news.ycombinator.com/context?id=39504104
Legend2440 3y ago
This is neither cheaper or simpler. Kernel methods are much more computationally expensive because the cost scales with the square of the dataset size. Neural networks are actually decently efficient, they mostly seem slow because we apply them to problems (like modeling the entire internet) that are just huge.

yazzku 3y ago

This seems like the key research to me if we want any shot at preventing the technology from being locked away behind big corp API walls interfacing giant data centers. Anything that removes the bloat and mysticism from the models so they can be scaled down and run on the little guy's computer is orders of magnitude more progress in my opinion than, e.g., increasing the token window by some epsilon.

westurner 3y ago

Does a given model converge after Gaussian blurring? What does it do in the presence of noise, given the curse of dimensionality? OpenCog integrates PLN and MOSES (~2005). "Interpretable Model-Based Hierarchical RL Using Inductive Logic Programming" (2021) https://news.ycombinator.com/item?id=37463686 https://news.ycombinator.com/item?id=37463686 : > https://en.wikipedia.org/wiki/Probabilistic_logic_network https://en.wikipedia.org/wiki/Probabilistic_logic_network : >> The basic goal of PLN is to provide reasonably accurate probabilistic inference in a way that is compatible with both term logic and predicate logic, and scales up to operate in real time on large dynamic knowledge bases Asmoses updates an in-RAM (*) online hypergraph with the graph relations it learns. CuPy wraps CuDNN. Re: quantum logic and quantum causal inference: https://news.ycombinator.com/item?id=38721246 https://news.ycombinator.com/item?id=38721246 From https://news.ycombinator.com/item?id=39255303 https://news.ycombinator.com/item?id=39255303 : > Is Quantum Logic the correct propositional logic? Is Quantum Logic a sufficient logic for all things? A quantum ML task: Find all local and nonlocal state linkages within the presented observations And then also do universal function approximation

westurner 3y ago
But also biological neurological systems; Coping strategies: https://en.wikipedia.org/wiki/Coping https://en.wikipedia.org/wiki/Coping Defense Mechanisms > Vaillant's categorization > Level 4: mature: https://en.wikipedia.org/wiki/Defence_mechanism#Level_4:_mature https://en.wikipedia.org/wiki/Defence_mechanism#Level_4:_mat... "From Comfort Zone to Performance Management" suggests that the Carnall coping cycle coincides with the TPR curve (Transforming, Performing, Reforming, adjourning); that coping with change in systems is linked with performance. And Consensus; social with nonlinear feedback and technological. What are the systems thinking advantages in such fields of study? Systems theory > See also > Glossary,: https://en.wikipedia.org/wiki/Systems_theory#See_also https://en.wikipedia.org/wiki/Systems_theory#See_also
- westurner 3y ago
  Wrong thread, my mistake; this comment was for this thread: https://news.ycombinator.com/context?id=39504104 https://news.ycombinator.com/context?id=39504104

Legend2440 3y ago

This is neither cheaper or simpler. Kernel methods are much more computationally expensive because the cost scales with the square of the dataset size. Neural networks are actually decently efficient, they mostly seem slow because we apply them to problems (like modeling the entire internet) that are just huge.