5 ms·

Rushton (1961) concluded that the neural signaling of a typical human myelinated nerve fiber spanning, say, between a finger and the spinal cord cannot employ a

Rushton (1961) concluded that the neural signaling of a typical human myelinated nerve fiber spanning, say, between a finger and the spinal cord cannot employ a continuous representation due to the presence of noise. Despite these seminal works, computational models based on continuous representation dominate present day neuroscience literature – for example, continuous attractor networks (Eliasmith, 2005; Wang, 2009). Sure, we know from basic information theory that any noisy system is inherently discrete. However, depending on the magnitude of the noise, I don't see why we should not describe models in terms of continuous variables. As an analogy, consider the average numerical computer program. We often derive the underlying math in terms of real-valued vector spaces. However, when implementing the program on a computer system, all variables are ultimately discrete. A reason for not thinking about our problems in terms of discrete objects is that math just tends to get incredibly complicated as soon as objects are discrete; and that the computational substrate is fine-grained enough (in most cases). Similarly, if noise limits the effective resolution of individual signals or representations in the brain to 5 bits (number taken from the paper), that (in my opinion) still does not mean that we should stop describing computational neuroscience models in continuous spaces -- at least not, if they are validated with the empirically measured amount of noise; which is exactly what Eliasmith's lab (cited above) does (full disclosure: I'm one of his students). Furthermore, as soon as you code information in populations of neurons, the noise on individual connections becomes less important; one could argue that whenever the brain needs precise computations, it dedicates more neural resources to that problem; then, the noise will "average out".

noobermin 8y ago
>I don't see why we should not describe models in terms of continuous variables. You're being way too polite in your wording, as your next paragraph states, every field that uses computers has done it since the computers were invented. Allow me to push it further: AFAIC almost all hand calculated math is discrete. The numbers you can write on a page are countable, in fact finite, so all actually calculated math is "discrete" in some sense. No one every really touches the full continuum of R other than abstractly: real calculations whenever you truncate pi or sqrt(2), or generally calculate with a fixed set of digits, you are doing _discrete_math_. So yes, Q is dense in R which helps, and you could write out more digits if you really wanted to, but even then, people restrict themselves to small, finite subsets of Q and even resorting to using things like logarithms/order of magnitude to keep that set as small as possible since our brains can't take all the hairiness. But still, that set is large enough to still do the things we care about anyway. However, when people want to think abstractly rather than explicitly (calculating), it's easier to take the limit as the gaps go to zero and deal with clean, C^\infty functions. I mean, that's literally what people mean (at least how my experimental physicist brain thinks it does) when they say analysis/calculus is about approximations. I deal with plasma in the hot, nonquantum limit so my ions and electrons are in fact discrete particles. Whenever I calculate an electric field or a magnetic field for the distribution, I ignore the fact that there will be noise on the scale of individual particles and replace that noisy function with a clean smooth function, which tends to be a good approximation. In fact, this is how almost every classical physics problem is solved, you ignore the structure on the particle level and pretend it is "continuous". Same with materials engineering, and so on.
- integration 8y ago
  Forgive my ignorance, but isn’t it true that we don’t know what elementary particles are made of? In other words, doesn’t it appear that matter is both continuous and discrete, and that we could concievably find particles that comprise elementary particles... and so on? Is there a name for this paradox?
  - noobermin 8y ago
    I mean, the standard model says they are fundamental, although there are theories that they may be "made up" of other things like in string theory, although those theories have (imo) struggled to compare to experiment or demand experiments that are infeasible today. Regarding "continuous and discrete", quantum mechanically, electrons aren't definitein space, you might be referring to that. In plasma physics we sit above the quantum limit (neglecting p and x variance or more correctly, the variances' product well exceed \hbar), so we treat them discretely, so in a sense, as all classical physics is, it's an approximation too. My point is that even in classical physics, I usually don't care about fluctuations on small scales which will be noisy, so on top of the classical approximation I make another approximation where I replace a noisy function with a smoother function that well approximates the noisy one. Smoothing out the noise is an important tool for theoretical understanding (as OP's student pointed out here), but it's important to remember it's just an approximation. EDIT: re the other replier. Another example is I treat ions as "fundamental" too, as we don't reach energies and conditions where their constituent nuclei matter, only ionization.
  - coldtea 8y ago
    Well, it's not a paradox as described. Just bad naming on our part, to prematurely call the first particles we discovered "elementary" without first waiting to see if they have more fundamental particles below them.
- nkozyra 8y ago
  > The numbers you can write on a page are countable, in fact finite, so all actually calculated math is "discrete" in some sense. Wouldn't it be more accurate to say you're simply working in a discrete _set_?
doomlaser 8y ago
> As an analogy, consider the average numerical computer program. We often derive the underlying math in terms of real-valued vector spaces. However, when implementing the program on a computer system, all variables are ultimately discrete. You're absolutely right, and this is a great analogy.
mzl 8y ago
> As an analogy, consider the average numerical computer program. We often derive the underlying math in terms of real-valued vector spaces. However, when implementing the program on a computer system, all variables are ultimately discrete. A reason for not thinking about our problems in terms of discrete objects is that math just tends to get incredibly complicated as soon as objects are discrete; and that the computational substrate is fine-grained enough (in most cases). When doing computations that require a lot of precision, one actually needs to consider the fact that floats/doubles are discrete and with limited precision, and handle that in the code. Typical examples may be numerical analysis, physical simulations, game world code (for assuring consistent state for all connected players), and geometrical algorithms. Fortunately for most programmers, they do not need that level of precision in their code.
myWindoonn 8y ago
What about the computable reals? Our computer programs are, in a sense, capable of emitting the digits of any computable real number, and we have arithmetic (Gosper continued-fraction arithmetic) on those digits. The fact that values might be discrete does not change that programs might be continuous over their output range; functions from computable reals to computable reals can be continuous.
- EGreg 8y ago
  Exactly. The question isn’t whether something is discrete so much as whether it is sampling a continuous function. Can you have arbitrarily large jumps, or not? See Buridan’s Principle :)

noobermin 8y ago

>I don't see why we should not describe models in terms of continuous variables. You're being way too polite in your wording, as your next paragraph states, every field that uses computers has done it since the computers were invented. Allow me to push it further: AFAIC almost all hand calculated math is discrete. The numbers you can write on a page are countable, in fact finite, so all actually calculated math is "discrete" in some sense. No one every really touches the full continuum of R other than abstractly: real calculations whenever you truncate pi or sqrt(2), or generally calculate with a fixed set of digits, you are doing _discrete_math_. So yes, Q is dense in R which helps, and you could write out more digits if you really wanted to, but even then, people restrict themselves to small, finite subsets of Q and even resorting to using things like logarithms/order of magnitude to keep that set as small as possible since our brains can't take all the hairiness. But still, that set is large enough to still do the things we care about anyway. However, when people want to think abstractly rather than explicitly (calculating), it's easier to take the limit as the gaps go to zero and deal with clean, C^\infty functions. I mean, that's literally what people mean (at least how my experimental physicist brain thinks it does) when they say analysis/calculus is about approximations. I deal with plasma in the hot, nonquantum limit so my ions and electrons are in fact discrete particles. Whenever I calculate an electric field or a magnetic field for the distribution, I ignore the fact that there will be noise on the scale of individual particles and replace that noisy function with a clean smooth function, which tends to be a good approximation. In fact, this is how almost every classical physics problem is solved, you ignore the structure on the particle level and pretend it is "continuous". Same with materials engineering, and so on.

integration 8y ago
Forgive my ignorance, but isn’t it true that we don’t know what elementary particles are made of? In other words, doesn’t it appear that matter is both continuous and discrete, and that we could concievably find particles that comprise elementary particles... and so on? Is there a name for this paradox?
- noobermin 8y ago
  I mean, the standard model says they are fundamental, although there are theories that they may be "made up" of other things like in string theory, although those theories have (imo) struggled to compare to experiment or demand experiments that are infeasible today. Regarding "continuous and discrete", quantum mechanically, electrons aren't definitein space, you might be referring to that. In plasma physics we sit above the quantum limit (neglecting p and x variance or more correctly, the variances' product well exceed \hbar), so we treat them discretely, so in a sense, as all classical physics is, it's an approximation too. My point is that even in classical physics, I usually don't care about fluctuations on small scales which will be noisy, so on top of the classical approximation I make another approximation where I replace a noisy function with a smoother function that well approximates the noisy one. Smoothing out the noise is an important tool for theoretical understanding (as OP's student pointed out here), but it's important to remember it's just an approximation. EDIT: re the other replier. Another example is I treat ions as "fundamental" too, as we don't reach energies and conditions where their constituent nuclei matter, only ionization.
- coldtea 8y ago
  Well, it's not a paradox as described. Just bad naming on our part, to prematurely call the first particles we discovered "elementary" without first waiting to see if they have more fundamental particles below them.
nkozyra 8y ago
> The numbers you can write on a page are countable, in fact finite, so all actually calculated math is "discrete" in some sense. Wouldn't it be more accurate to say you're simply working in a discrete _set_?

doomlaser 8y ago

mzl 8y ago

> As an analogy, consider the average numerical computer program. We often derive the underlying math in terms of real-valued vector spaces. However, when implementing the program on a computer system, all variables are ultimately discrete. A reason for not thinking about our problems in terms of discrete objects is that math just tends to get incredibly complicated as soon as objects are discrete; and that the computational substrate is fine-grained enough (in most cases). When doing computations that require a lot of precision, one actually needs to consider the fact that floats/doubles are discrete and with limited precision, and handle that in the code. Typical examples may be numerical analysis, physical simulations, game world code (for assuring consistent state for all connected players), and geometrical algorithms. Fortunately for most programmers, they do not need that level of precision in their code.

myWindoonn 8y ago

What about the computable reals? Our computer programs are, in a sense, capable of emitting the digits of any computable real number, and we have arithmetic (Gosper continued-fraction arithmetic) on those digits. The fact that values might be discrete does not change that programs might be continuous over their output range; functions from computable reals to computable reals can be continuous.

EGreg 8y ago
Exactly. The question isn’t whether something is discrete so much as whether it is sampling a continuous function. Can you have arbitrarily large jumps, or not? See Buridan’s Principle :)