MAXPOOL

13 ms·

91.
Efficient Transformers: A Survey(arxiv.org)
2 points by MAXPOOL 6y ago|0 comments
92.
Visualizing the Structure of a Neural Network(youtube.com)
1 points by MAXPOOL 6y ago|0 comments
93.
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence(arxiv.org)
1 points by MAXPOOL 6y ago|0 comments
94.
by MAXPOOL 6y ago
> We show that the transformer attention mechanism is the update rule of a modern Hopfield network with continuous states. This new Hopfield network can store exponentially (with the dimension) many patterns, converges with one update, a
95.
Hopfield Networks Is All You Need(arxiv.org)
4 points by MAXPOOL 6y ago|1 comments
96.
Caltech Letters(caltechletters.org)
2 points by MAXPOOL 6y ago|0 comments
97.
by MAXPOOL 6y ago
You are probably right. Neural doohickeys find real uses in computational chemistry. Accurate ab initio solvers (many-body Schrödinger equation) are very slow and everything else is just heuristic-, quasi- approximation mixed with semi-empi
98.
The Computational Limits of Deep Learning(arxiv.org)
1 points by MAXPOOL 6y ago|0 comments
99.
by MAXPOOL 6y ago
>In the general sense, is out-of-distribution learning even possible? Yes. Humans can learn to answer questions even when questions (test cases) don't come from the same distribution as the training set. You need to have some amoun
100.
by MAXPOOL 6y ago
The representation GPT-3 learns are flexible. It can learn very complex tasks, including logic and simple arithmetic. The issue is GPT-3 learning algorithm and it's learning capability. Typically the ability to learn << ability
101.
by MAXPOOL 6y ago
> I don't think that you can conclude that humans learn more efficiency based on just quantity of data. You are correct. My main argument was that in-distribution learning is not enough. You can's fix that problem with more dat
102.
by MAXPOOL 6y ago
That's true. I don't think humans are very good at general intelligence. Our genes give us priors to evade lions and pick berries. Humanity as cultural organism can reprogram us to more abstract tasks, but it takes easily 10-20 y
103.
by MAXPOOL 6y ago
Even in the terms of total human sensory input data rate, humans learn from a very tiny data set. 16 year old human has experienced only about 400 million wakeful seconds.
104.
by MAXPOOL 6y ago
A great demonstration of why in-distribution learning is not enough for AGI. The largest GPT-3 model "GPT-3 175B" is trained with 499 Billion tokens (one token is maybe equal to 4 characters in text). Human reading/talking&#x
105.
Graph Representation Learning and Beyond (GRL+) ICML 2020 Workshop(grlplus.github.io)
3 points by MAXPOOL 6y ago|0 comments
106.
Eye-catching advances in some AI fields are not real(sciencemag.org)
5 points by MAXPOOL 6y ago|0 comments
107.
ICLR2020: The Future Is Self-Supervised (LeCun and Bengio)(iclr.cc)
2 points by MAXPOOL 6y ago|0 comments
108.
by MAXPOOL 6y ago
Modern consensus is that 4 is the real magic number. Humans can store only four chunks in short-term memory tasks. For working memory and visual memory the number 4 pops up again and again. When you are holding 5-9 things in memory you
109.
by MAXPOOL 7y ago
39 bits/s is: - 1.230 Gb/year - 24 Gb in 20 years = 3 GB 3 GB is theoretical upper limit and overestimate to what human can learn verbally until they are 20 years old. It would require being awake and communicating 24/hour
110.
by MAXPOOL 7y ago
I am just an NPC until I get coffee. For AI researcher Game AI is like porn. It's cheap tricks and obviously fake but oddly fascinating. Sometimes you find a new trick you want to try in real life. Marvin Minsky once said "I bet
111.
by MAXPOOL 7y ago
You can make AlphaZero learn different value function by changing the reward from the game. Game with no reward from draw, more reward from shorter games with more captures creates different value functions.
112.
by MAXPOOL 7y ago
It would be interesting to see AlphaZero trained to play very aggressive romantic style. AlphaImmortal's goal should be beating Magnus Carlsen 51% of the time with the most amazing games ever played. I think you could do this by chang
113.
by MAXPOOL 7y ago
I agree with you in principle. I still think reproducible should be a goal even in pure research in blue sky machine learning for the following reasons: Even basic research is can be sensitive to omitted parameters, setups and starting cond
114.
by MAXPOOL 7y ago
Humanity as a whole as problem solving algorithm might be similar to particle swarm in very complex neighbourhoods and topologies. Particles (individual humans or small groups) move around in problem space as particles with position and vel
115.
by MAXPOOL 7y ago
NeurIPS should be about fundamental science not about applied science and engineering solutions for specific domains. Science vs applications would be actually a good way to divide NeurIPS into two conferences to get focus back: NeurIPS and
116.
Survival: The First 3.8B Years(nature.com)
2 points by MAXPOOL 7y ago|0 comments
117.
by MAXPOOL 7y ago
Summary: Only brain activity involving ‘L5p neurons’ enters conscious awareness Is this brain cell your ‘mind’s eye’? https://blog.frontiersin.org/2019/10/01/systems-neuroscience...
118.
Coupling the State and Contents of Consciousness(frontiersin.org)
1 points by MAXPOOL 7y ago|1 comments
119.
by MAXPOOL 7y ago
Logic is usually considered as a part of philosophy. There are people in philosophy departments studying formal logic. I think all formal sciences should be considered as part of philosophy because they provide tools and language for the
120.
No one knows what attention is(link.springer.com)
4 points by MAXPOOL 7y ago|0 comments

More ›