8 ms·

Here is the reasoning: https://www.nobelprize.org/uploads/2024/09/advanced-physicsprize2024.pdf https://www.nobelprize.org/uploads/2024/09/advanced-physicsp...

Here is the reasoning: https://www.nobelprize.org/uploads/2024/09/advanced-physicsprize2024.pdf https://www.nobelprize.org/uploads/2024/09/advanced-physicsp... I'm surprised Terry Sejnowski isn't included, considering it seems to be for Hopfield Nets and Boltzmann machines, where Terry played a large role in the latter.

whizzter 2y ago
He was probably considered since he is mentioned in the reasoning paper, still it could be one of those unfortunate omissions in the nobel history since those deciding the prize might have a hard time to measure impact.
- gtirloni 2y ago
  Then they shouldn't be trusted to give awards in an area they are not experts in.
  - whizzter 2y ago
    That would probably leave the prizes awarded in a very narrow field, also the prize is supposed to be given to the thing that has "conferred the greatest benefit to humankind". So in this case they picked something that might be viewed as only having a tangential connection to the field, but the impact has been so immense that they probably went outside their regular comfort zone (and how many prizes can we give for LHC work that really don't touch regular human lives in the foreseeable future anyhow?).
  - bjoli 2y ago
    Impact is hard to quantify. There have been several occasions where someone who very well deserved a Nobel prize didn't get one. There are all kinds of reasons. Given he is mentioned in the reasoning he was probably considered. We can't know the reason he did not get the prize. I recently watched this quite video on the subject: https://youtu.be/zS7sJJB7BUI?feature=shared https://youtu.be/zS7sJJB7BUI?feature=shared and found it quite enjoyable.
    - moomin 2y ago
      A funny one is that there was so much objection to General Relativity that they compromised by giving Einstein a Nobel for his work on quantum mechanics. His acceptance speech was about general relativity.
  - PokemonNoGo 2y ago
    Shouldn't be trusted? They are a random swedish foundation. How is one going to change that? Or disallow it or what you want.
  - henrikschroder 2y ago
    Trusted? The will of Alfred Nobel states that the Royal Swedish Academy of Sciences is the body that selects the winner, you can't change that. Also, I think the process looks fairly decent: https://www.nobelprize.org/nomination/physics/ https://www.nobelprize.org/nomination/physics/ Gather nominations, make a shortlist, research the shortlist with actual field experts, present candidates, discuss, and vote. And in 50 years you'll be able to find out who the other candidates were!
  - brookst 2y ago
    This is where passive voice highlights a weak position. Trusted… by who?
  - learningstud 2y ago
    Don't indulge in meritocracy. It's essentially a "prize".
KingFelix 2y ago
Yeah, Terry is a rockstar, and pumping out tons of papers. Maybe they google scholared, list by citations?
dang 2y ago
I guess it makes sense to use that link above since it goes into much more detail. Changed from https://www.nobelprize.org/prizes/physics/2024/summary/ https://www.nobelprize.org/prizes/physics/2024/summary/. Thanks!
- thrdbndndn 2y ago
  I know it's one day late but I personally don't agree with this change. Using the main entry webpage is much easier for people to look for relevant info, including but not limited to: various articles about this specific award; other info about this year's Nobel, other info about Nobel in general. This is exactly what the Internet (or WWW) is better than traditional printing media. Using a PDF as a link (which itself can be easily found from the entry point) defeats it.
  - dang 2y ago
    I wish they were, but I don't think most readers are as diligent as you in looking for other sources.
mistercheph 2y ago
Whether or not these fields are meaningfully distinct is a matter of taste, despite the fashion being to imagine a plurality of interconnected but autonomous domains.
hxnamer 2y ago
Is this a widely accepted version of neural network history? I recognize Rosenblatt, Perceptron, etc., but I have never heard that Hopfield nets or Bolzmann machines were given any major weight in the history. The descriptions I have read were all mathematical, focusing on the computational graph with the magical backpropagation (which frankly is just memoizing intermediate computations). The descriptions also went out of their way to discourage terms like "synapses" and rather use "units".
- Onavo 2y ago
  The deep learning variety of neural networks are heavily simplified, mostly linear versions of biological neurons. They don't resemble anything between your ears. Real life neurons are generally modeled by differential equations (in layman terms, have many levels of feedback loops tied to time), not the simplified ones used in dense layer activation functions. Here are some examples https://snntorch.readthedocs.io/en/latest/tutorials/tutorial_1.html https://snntorch.readthedocs.io/en/latest/tutorials/tutorial...
  - KoolKat23 2y ago
    Would that be equivalent to Weight matrice parameters?
    - Onavo 2y ago
      Ish, take a look at the curves of the spiking neural network function, they are very different from the deep learning nets. When we "model" biological neural nets in code, we are essentially coming up with a mathematical transfer function that can replicate the chemical gradient changes and electrical impulses of a real neuron. Imagine playing a 3D computer game like Minecraft, the physics is not perfect but they are "close enough" to the real world.
      - KoolKat23 2y ago
        Thanks this is very insightful. Would the multi-head attention (Wv) not be equivalent to the chemical gradient changes? (there are multiple matrices in multi-head attention, one for each attention head and what I imagine would be the equivalent of different gradients This allows each attention head to learn different representations and focus on different aspects of the input sequence.) And then the output produced after applying the concatenated (W0 or output projection), be equivalent to the different electrical outputs such as the spikes and passed to the next neuron equivalent or attention head?
seydor 2y ago
Second time he gets overlooked (after turing award)
soheil 2y ago
Are you surprised every year that he's not included?

whizzter 2y ago

He was probably considered since he is mentioned in the reasoning paper, still it could be one of those unfortunate omissions in the nobel history since those deciding the prize might have a hard time to measure impact.

gtirloni 2y ago
Then they shouldn't be trusted to give awards in an area they are not experts in.
- whizzter 2y ago
  That would probably leave the prizes awarded in a very narrow field, also the prize is supposed to be given to the thing that has "conferred the greatest benefit to humankind". So in this case they picked something that might be viewed as only having a tangential connection to the field, but the impact has been so immense that they probably went outside their regular comfort zone (and how many prizes can we give for LHC work that really don't touch regular human lives in the foreseeable future anyhow?).
- bjoli 2y ago
  Impact is hard to quantify. There have been several occasions where someone who very well deserved a Nobel prize didn't get one. There are all kinds of reasons. Given he is mentioned in the reasoning he was probably considered. We can't know the reason he did not get the prize. I recently watched this quite video on the subject: https://youtu.be/zS7sJJB7BUI?feature=shared https://youtu.be/zS7sJJB7BUI?feature=shared and found it quite enjoyable.
  - moomin 2y ago
    A funny one is that there was so much objection to General Relativity that they compromised by giving Einstein a Nobel for his work on quantum mechanics. His acceptance speech was about general relativity.
- PokemonNoGo 2y ago
  Shouldn't be trusted? They are a random swedish foundation. How is one going to change that? Or disallow it or what you want.
- henrikschroder 2y ago
  Trusted? The will of Alfred Nobel states that the Royal Swedish Academy of Sciences is the body that selects the winner, you can't change that. Also, I think the process looks fairly decent: https://www.nobelprize.org/nomination/physics/ https://www.nobelprize.org/nomination/physics/ Gather nominations, make a shortlist, research the shortlist with actual field experts, present candidates, discuss, and vote. And in 50 years you'll be able to find out who the other candidates were!
- brookst 2y ago
  This is where passive voice highlights a weak position. Trusted… by who?
- learningstud 2y ago
  Don't indulge in meritocracy. It's essentially a "prize".

KingFelix 2y ago

Yeah, Terry is a rockstar, and pumping out tons of papers. Maybe they google scholared, list by citations?

dang 2y ago

I guess it makes sense to use that link above since it goes into much more detail. Changed from https://www.nobelprize.org/prizes/physics/2024/summary/ https://www.nobelprize.org/prizes/physics/2024/summary/. Thanks!

thrdbndndn 2y ago
I know it's one day late but I personally don't agree with this change. Using the main entry webpage is much easier for people to look for relevant info, including but not limited to: various articles about this specific award; other info about this year's Nobel, other info about Nobel in general. This is exactly what the Internet (or WWW) is better than traditional printing media. Using a PDF as a link (which itself can be easily found from the entry point) defeats it.
- dang 2y ago
  I wish they were, but I don't think most readers are as diligent as you in looking for other sources.

mistercheph 2y ago

Whether or not these fields are meaningfully distinct is a matter of taste, despite the fashion being to imagine a plurality of interconnected but autonomous domains.

hxnamer 2y ago

Is this a widely accepted version of neural network history? I recognize Rosenblatt, Perceptron, etc., but I have never heard that Hopfield nets or Bolzmann machines were given any major weight in the history. The descriptions I have read were all mathematical, focusing on the computational graph with the magical backpropagation (which frankly is just memoizing intermediate computations). The descriptions also went out of their way to discourage terms like "synapses" and rather use "units".

Onavo 2y ago
The deep learning variety of neural networks are heavily simplified, mostly linear versions of biological neurons. They don't resemble anything between your ears. Real life neurons are generally modeled by differential equations (in layman terms, have many levels of feedback loops tied to time), not the simplified ones used in dense layer activation functions. Here are some examples https://snntorch.readthedocs.io/en/latest/tutorials/tutorial_1.html https://snntorch.readthedocs.io/en/latest/tutorials/tutorial...
- KoolKat23 2y ago
  Would that be equivalent to Weight matrice parameters?
  - Onavo 2y ago
    Ish, take a look at the curves of the spiking neural network function, they are very different from the deep learning nets. When we "model" biological neural nets in code, we are essentially coming up with a mathematical transfer function that can replicate the chemical gradient changes and electrical impulses of a real neuron. Imagine playing a 3D computer game like Minecraft, the physics is not perfect but they are "close enough" to the real world.
    - KoolKat23 2y ago
      Thanks this is very insightful. Would the multi-head attention (Wv) not be equivalent to the chemical gradient changes? (there are multiple matrices in multi-head attention, one for each attention head and what I imagine would be the equivalent of different gradients This allows each attention head to learn different representations and focus on different aspects of the input sequence.) And then the output produced after applying the concatenated (W0 or output projection), be equivalent to the different electrical outputs such as the spikes and passed to the next neuron equivalent or attention head?

seydor 2y ago

Second time he gets overlooked (after turing award)

soheil 2y ago

Are you surprised every year that he's not included?