5 ms·

> The article repeatedly makes the mistaken assertion that sending a 'Yo' constitutes 1 bit of information transmitted. This is mistaken as the information also

by hk__2 6y ago

> The article repeatedly makes the mistaken assertion that sending a 'Yo' constitutes 1 bit of information transmitted. This is mistaken as the information also holds temporal information, which is actually what is very useful. t's not a Yo at some random point, it's a Yo now. It's really encoding "now!". Still not very much information but a ton more than 1 bit. It’s still one bit. The time is not part of the transmission.

capableweb 6y ago
Time is absolutely part of the transmission. It matters when you say "Hi" to someone. Differently timed "Hi"'s can say different things. Even if you repeat "Hi" many times, it's still just "Hi" but the amount of them also matters and becomes part of the transmission when looking at all of them together. I feel like many, especially developers, fail to see what is in between the data to matter as well as the data itself. Seems like a common hole to fall into.
- admax88q 6y ago
  If we're in different locations and we run a wire between us, and I send a single pulse down the wire. That's one bit of information. You as the receiver derive the other relevant information from that. A timestamp was not encoded and transmitted, a single bit (pulse) was transmitted from me to you. That's the thesis of the article I believe. That single bit can "carry" a lot of additional information due to the context and understanding of the receiver. Information that wasn't actually transmitted in the message, but derived from it.
  - SamReidHughes 6y ago
    That’s not one bit of information. If there is only one possible message you can send, there’s zero bits in the message itself. One bit would be if you had two messages to choose from. All the information in Yo is in the timing.
    - hk__2 6y ago
      Timing is the context, not the information actually transmitted.
    - StavrosK 6y ago
      You do have two messages to choose from. One is no message, the other is a message.
      - SamReidHughes 6y ago
        In that case you have 1 bit per nanosecond of information, or however frequently you want to model sampling the message, not 1 bit of information. I mean ultimately you need to somehow map the sequence of Yo messages into "information" in a fairly precise manner. Receiving the message takes you from one probability distribution about the world -- or some particular question about the world -- to a smaller probability distribution. I'd describe the information the message carries based on that. It's contingent on what the question is.
- stevenhuang 6y ago
  More accurate to say time can be derived from the transmission. It's still not a part of it. Your latter contribution misses the mark by failing to consider that you all might just be talking past each other, arguing over semantics. That's the most common hole to fall into. It's a 1 bit transmission with the ever-present time side-channel.
- tsimionescu 6y ago
  You're arguing past each other. The amount of information transmitted is 1 bit. Someone receiving a PCAP with the data would not get more than 1 bit of information. However, you're also right that the people sending and receiving the bit can derive more information from it.
  - thaumasiotes 6y ago
    > You're arguing past each other. The amount of information transmitted is 1 bit. I don't understand the argument here. In order to get up to one bit of information, you need to take account of the fact that "yo" is only sent at particular times. If you ignore that fact, sending "yo" provides exactly zero bits of information, because it is the only possible message. All of the information content comes from the existence of the message; none of it comes from the contents. So in order to claim that there's any information present at all, you have to admit that time of sending carries information. The amount of information carried then depends on the duration over which messages might have been transmitted, since that possibility is what imbues information into messages that are transmitted. It also imbues information into messages that aren't transmitted, unless you believe that a binary message of 01000110 transmits just 3, and not 8, bits of information.
    - stevenhuang 6y ago
      It's called a side channel. The amount of information _sent_ really is 1 bit.
      - thaumasiotes 6y ago
        > The amount of information _sent_ really is 1 bit. I'm not sure how you're calculating this, but it is obviously wrong. The information conveyed by the contents of a message is the log of the number of possible messages. Here, there is one possible message. The log of one is zero. There's a big difference between one bit of information and zero. You can interpret a "yo" message as conveying one bit of information, and the only way to do that is to draw it from the contrast between "sending a yo" and "not sending a yo". If you do that, "sending a yo" sends one bit of information, and "not sending a yo" also sends one bit of information. Thought experiment: with three bits of information, you can disambiguate between eight possibilities. Try to design a protocol in which, by sending up to three "yo"s, you achieve 8 distinct messages.
ericjang 6y ago
It is widely believed by neuroscientists that spike timing in animal brains (whether by aggregate rate or by individual spikes) does the majority of information encoding.
wave_function 6y ago
I’d argue that choosing the time to send a “yo” constitutes log2(N) bits, where N is the number of times the yo could have been sent (It’s not quite an integer but you might say resolution faster than a minute is irrelevant and call N the number of minutes). Not much, but still more than one.

by hk__2 6y ago

capableweb 6y ago

Time is absolutely part of the transmission. It matters when you say "Hi" to someone. Differently timed "Hi"'s can say different things. Even if you repeat "Hi" many times, it's still just "Hi" but the amount of them also matters and becomes part of the transmission when looking at all of them together. I feel like many, especially developers, fail to see what is in between the data to matter as well as the data itself. Seems like a common hole to fall into.

admax88q 6y ago
If we're in different locations and we run a wire between us, and I send a single pulse down the wire. That's one bit of information. You as the receiver derive the other relevant information from that. A timestamp was not encoded and transmitted, a single bit (pulse) was transmitted from me to you. That's the thesis of the article I believe. That single bit can "carry" a lot of additional information due to the context and understanding of the receiver. Information that wasn't actually transmitted in the message, but derived from it.
- SamReidHughes 6y ago
  That’s not one bit of information. If there is only one possible message you can send, there’s zero bits in the message itself. One bit would be if you had two messages to choose from. All the information in Yo is in the timing.
  - hk__2 6y ago
    Timing is the context, not the information actually transmitted.
  - StavrosK 6y ago
    You do have two messages to choose from. One is no message, the other is a message.
    - SamReidHughes 6y ago
      In that case you have 1 bit per nanosecond of information, or however frequently you want to model sampling the message, not 1 bit of information. I mean ultimately you need to somehow map the sequence of Yo messages into "information" in a fairly precise manner. Receiving the message takes you from one probability distribution about the world -- or some particular question about the world -- to a smaller probability distribution. I'd describe the information the message carries based on that. It's contingent on what the question is.
stevenhuang 6y ago
More accurate to say time can be derived from the transmission. It's still not a part of it. Your latter contribution misses the mark by failing to consider that you all might just be talking past each other, arguing over semantics. That's the most common hole to fall into. It's a 1 bit transmission with the ever-present time side-channel.
tsimionescu 6y ago
You're arguing past each other. The amount of information transmitted is 1 bit. Someone receiving a PCAP with the data would not get more than 1 bit of information. However, you're also right that the people sending and receiving the bit can derive more information from it.
- thaumasiotes 6y ago
  > You're arguing past each other. The amount of information transmitted is 1 bit. I don't understand the argument here. In order to get up to one bit of information, you need to take account of the fact that "yo" is only sent at particular times. If you ignore that fact, sending "yo" provides exactly zero bits of information, because it is the only possible message. All of the information content comes from the existence of the message; none of it comes from the contents. So in order to claim that there's any information present at all, you have to admit that time of sending carries information. The amount of information carried then depends on the duration over which messages might have been transmitted, since that possibility is what imbues information into messages that are transmitted. It also imbues information into messages that aren't transmitted, unless you believe that a binary message of 01000110 transmits just 3, and not 8, bits of information.
  - stevenhuang 6y ago
    It's called a side channel. The amount of information _sent_ really is 1 bit.
    - thaumasiotes 6y ago
      > The amount of information _sent_ really is 1 bit. I'm not sure how you're calculating this, but it is obviously wrong. The information conveyed by the contents of a message is the log of the number of possible messages. Here, there is one possible message. The log of one is zero. There's a big difference between one bit of information and zero. You can interpret a "yo" message as conveying one bit of information, and the only way to do that is to draw it from the contrast between "sending a yo" and "not sending a yo". If you do that, "sending a yo" sends one bit of information, and "not sending a yo" also sends one bit of information. Thought experiment: with three bits of information, you can disambiguate between eight possibilities. Try to design a protocol in which, by sending up to three "yo"s, you achieve 8 distinct messages.

ericjang 6y ago

It is widely believed by neuroscientists that spike timing in animal brains (whether by aggregate rate or by individual spikes) does the majority of information encoding.

wave_function 6y ago

I’d argue that choosing the time to send a “yo” constitutes log2(N) bits, where N is the number of times the yo could have been sent (It’s not quite an integer but you might say resolution faster than a minute is irrelevant and call N the number of minutes). Not much, but still more than one.