6 ms·

The Tsetlin Machine outperforms neural networks

139 points by beagle3 8y ago

kurthr 8y ago
Just go straight to the paper or code... the write-up is almost content free. Paper: https://arxiv.org/abs/1804.01508 https://arxiv.org/abs/1804.01508 Code: https://github.com/cair/TsetlinMachine https://github.com/cair/TsetlinMachine
xtacy 8y ago
/r/MachineLearning comments on the same paper from April 2018: https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_the_tsetlin_machine_a_new_approach_to_ml/ https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_t...
- wetpaws 8y ago
  See also: https://www.reddit.com/r/MachineLearning/comments/ab8na3/project_new_tsetlin_machine_implementation_with/ https://www.reddit.com/r/MachineLearning/comments/ab8na3/pro...
nl 8y ago
The paper is actually pretty interesting - it's a pity the headline is misleading. The Reddit comment[1] is a good summary: More accurately: "The Tsetlin Machine - a new approach to ML - outperforms single layer Neural Networks, SVMs, Random Forests, the Naive Bayes Classifier and Logistic Regression on four carefully selected contrived datasets" Notably, on their (weird!) "Binary Iris" dataset, the NN appears to be undertrained, and it is unclear what the headline "mean" accuracy figure is actually a mean of. However, once we get over that, it's quite a different approach, and I can imagine places where it could be useful. Notably, as an anomaly detector it would seem to have interesting properties like interpretable results similar to a random forest. [1] https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_the_tsetlin_machine_a_new_approach_to_ml/dwujoxt/ https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_t...
- nestorD 8y ago
  They say that their results are interpretable due to being logical formulas but, later, show a toy problem in which their formula has 10000 clauses. By that definition deep-learning is probably interpretable. However I do think that it is an interesting concept that would probably be worth testing if your inputs and outputs have a natural mapping to booleans.
simonw 8y ago
The entire implementation appears to be 300 lines of Cython: https://github.com/cair/TsetlinMachineCython/blob/master/TsetlinMachine.pyx https://github.com/cair/TsetlinMachineCython/blob/master/Tse...
nestorD 8y ago
The Tsetlin Machine being a specialized method, it makes sense that it beats neural networks in its domain of predilection (decision from, potentially noisy, binary inputs).
chvid 8y ago
So what is a Tselin machine? Maybe I am lazy but a don’t see an exact description. I am assume it is a function that takes an input, a state and produces an output. But what exactly is it? A neural net is a fairly general construction. Is this not just a nn with a particular activation function, a particular network layout?
- shakna 8y ago
  For a detailed understanding, you need the paper. [0] But, loosely speaking, it is a classifier that works best with less information, utilising a series of formulas. It only really does bitwise ops for recognition, and is easy to implement and fast. If I were to describe it in more flowery terms, hashing and finite automata had a baby. [0] https://arxiv.org/abs/1804.01508 https://arxiv.org/abs/1804.01508
  - chvid 8y ago
    Why can't I just get a reasonable definition of what a Tsetlin machine is? As far as I can read it a function that takes an input (a vector of real numbers in 0..1 (?)), a state (in form of an integer) and produces an output (in form of something?).
    - brownbat 8y ago
      What constitutes reasonable? Here's an example of someone explaining something to five different levels of audiences. https://www.youtube.com/watch?v=OWJCfOvochA https://www.youtube.com/watch?v=OWJCfOvochA None of these approaches are prima facie "unreasonable." But if you get the wrong level it won't feel useful. I don't think it's crazy to have open teaching discussions on HN, I think you lay out a fair hope that someone can break something down here. But I'm just not sure which level you're looking for. Precision and simplicity are often competing constraints.
      - chvid 8y ago
        Okay. Let me try getting my point across with an insult instead. Some dude in a marginal university in Norway has managed to convince a few people, a couple of students and a writer for a glittery university magazine, that he is a genius. His approach is to do something trivial but bury it in a completely obscure terminology. This technically sorta works on a toy example but not really on more challenging tests. But it is very convincing for people who are not trying to understand what is going on but rather are impressed by technical words and academic rank. That is why it is getting negative feedback here and on other technical forums. And that is why noone can answer a simple technical question.
alexyak 8y ago
The Tsetlin machine is based on Tsetlin’s automata, the so called finite automata with linear tactics. The are basically counters of rewards (up) and penalties (down) with various strategies of integration, proportionization and differentiation. They have minimum necessary for digital emulation of the perception functionality. They str not only due to Tsetlin but also yo Krinsky, Krylov, Varshavsky. Mikhail Tsetlin is the most prominent creator. It’s the work in the USSR in the 60s. So not new but very elegant!

139 points by beagle3 8y ago

Just go straight to the paper or code... the write-up is almost content free. Paper: https://arxiv.org/abs/1804.01508 https://arxiv.org/abs/1804.01508 Code: https://github.com/cair/TsetlinMachine https://github.com/cair/TsetlinMachine

xtacy 8y ago

/r/MachineLearning comments on the same paper from April 2018: https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_the_tsetlin_machine_a_new_approach_to_ml/ https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_t...

wetpaws 8y ago
See also: https://www.reddit.com/r/MachineLearning/comments/ab8na3/project_new_tsetlin_machine_implementation_with/ https://www.reddit.com/r/MachineLearning/comments/ab8na3/pro...

nl 8y ago

The paper is actually pretty interesting - it's a pity the headline is misleading. The Reddit comment[1] is a good summary: More accurately: "The Tsetlin Machine - a new approach to ML - outperforms single layer Neural Networks, SVMs, Random Forests, the Naive Bayes Classifier and Logistic Regression on four carefully selected contrived datasets" Notably, on their (weird!) "Binary Iris" dataset, the NN appears to be undertrained, and it is unclear what the headline "mean" accuracy figure is actually a mean of. However, once we get over that, it's quite a different approach, and I can imagine places where it could be useful. Notably, as an anomaly detector it would seem to have interesting properties like interpretable results similar to a random forest. [1] https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_the_tsetlin_machine_a_new_approach_to_ml/dwujoxt/ https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_t...

nestorD 8y ago
They say that their results are interpretable due to being logical formulas but, later, show a toy problem in which their formula has 10000 clauses. By that definition deep-learning is probably interpretable. However I do think that it is an interesting concept that would probably be worth testing if your inputs and outputs have a natural mapping to booleans.

simonw 8y ago

The entire implementation appears to be 300 lines of Cython: https://github.com/cair/TsetlinMachineCython/blob/master/TsetlinMachine.pyx https://github.com/cair/TsetlinMachineCython/blob/master/Tse...

nestorD 8y ago

The Tsetlin Machine being a specialized method, it makes sense that it beats neural networks in its domain of predilection (decision from, potentially noisy, binary inputs).

chvid 8y ago

So what is a Tselin machine? Maybe I am lazy but a don’t see an exact description. I am assume it is a function that takes an input, a state and produces an output. But what exactly is it? A neural net is a fairly general construction. Is this not just a nn with a particular activation function, a particular network layout?

shakna 8y ago
For a detailed understanding, you need the paper. [0] But, loosely speaking, it is a classifier that works best with less information, utilising a series of formulas. It only really does bitwise ops for recognition, and is easy to implement and fast. If I were to describe it in more flowery terms, hashing and finite automata had a baby. [0] https://arxiv.org/abs/1804.01508 https://arxiv.org/abs/1804.01508
- chvid 8y ago
  Why can't I just get a reasonable definition of what a Tsetlin machine is? As far as I can read it a function that takes an input (a vector of real numbers in 0..1 (?)), a state (in form of an integer) and produces an output (in form of something?).
  - brownbat 8y ago
    What constitutes reasonable? Here's an example of someone explaining something to five different levels of audiences. https://www.youtube.com/watch?v=OWJCfOvochA https://www.youtube.com/watch?v=OWJCfOvochA None of these approaches are prima facie "unreasonable." But if you get the wrong level it won't feel useful. I don't think it's crazy to have open teaching discussions on HN, I think you lay out a fair hope that someone can break something down here. But I'm just not sure which level you're looking for. Precision and simplicity are often competing constraints.
    - chvid 8y ago
      Okay. Let me try getting my point across with an insult instead. Some dude in a marginal university in Norway has managed to convince a few people, a couple of students and a writer for a glittery university magazine, that he is a genius. His approach is to do something trivial but bury it in a completely obscure terminology. This technically sorta works on a toy example but not really on more challenging tests. But it is very convincing for people who are not trying to understand what is going on but rather are impressed by technical words and academic rank. That is why it is getting negative feedback here and on other technical forums. And that is why noone can answer a simple technical question.

alexyak 8y ago

The Tsetlin machine is based on Tsetlin’s automata, the so called finite automata with linear tactics. The are basically counters of rewards (up) and penalties (down) with various strategies of integration, proportionization and differentiation. They have minimum necessary for digital emulation of the perception functionality. They str not only due to Tsetlin but also yo Krinsky, Krylov, Varshavsky. Mikhail Tsetlin is the most prominent creator. It’s the work in the USSR in the 60s. So not new but very elegant!