3 ms·

Hundreds of kilobytes is not a lot of information. Even guessing gigabytes would be going very controversially low. Just the delta versus some "average" human g

by jerf 3y ago

Hundreds of kilobytes is not a lot of information. Even guessing gigabytes would be going very controversially low. Just the delta versus some "average" human genome would easily be into the hundreds kilobytes. Even just modeling a human as the sum total of their sensory input, itself a simplification transparently obviously insufficient, hundreds of kilobytes covers just a few seconds of their existence, or with compression, minutes or maybe hours, tops. Imagine trying to create a Marcus Aurelius that correctly output exactly the contents of his writing, while also being acceptably a human analog. If you think you can stick that into a few hundred kilobytes, there's a 500,000 Euro prize you should be starting forthwith as you're claiming many orders of magnitude improvement over the current best: http://prize.hutter1.net/ http://prize.hutter1.net/

addaon 3y ago
> Hundreds of kilobytes is not a lot of information. Humans are very, very predictable. It's not clear that the deviation of one human from the mean has a lot of information. It's not clear that it /doesn't/, either -- but I'd sure as heck take the under on gigabytes for novelty of text output. Keep in mind that we're not trying to model the internal state of a human machine; instead, we're discussing how much additional information is needed to align the output of a generic human text generator with the output of a specific individual. A lifetime is about two billion seconds -- so a gigabyte is four bits of data /per second/, or maybe two bits of data /per word/ for someone who continuously speaks for 70 years. Derate to a reasonable percentage of a lifetime spent generating output (10%? 1%?) and you're talking many, many bits per word. That's enough to Huffman compress their entire lifetime output with the "base model" entirely a dictionary, no need to even have a generative model!
- jerf 3y ago
  "Keep in mind that we're not trying to model the internal state of a human machine" Yes, that is exactly what we are discussing in this thread: Modelling the internal state of the human machine. The discussion of generating the exact stream of words is otherwise pointless. Fire lz at the collected output and call it a day; it isn't optimal but it's close enough. The entire conversation is whether we could recover the human consciousness from just the historical record.
csours 3y ago
> Hundreds of kilobytes is not a lot of information. But how much work is done by my mind when I read the output of the machine? It seems like humans are very strongly soft-wired (if not hard-wired, whatever that means) to try to understand other humans.

by jerf 3y ago

addaon 3y ago

> Hundreds of kilobytes is not a lot of information. Humans are very, very predictable. It's not clear that the deviation of one human from the mean has a lot of information. It's not clear that it /doesn't/, either -- but I'd sure as heck take the under on gigabytes for novelty of text output. Keep in mind that we're not trying to model the internal state of a human machine; instead, we're discussing how much additional information is needed to align the output of a generic human text generator with the output of a specific individual. A lifetime is about two billion seconds -- so a gigabyte is four bits of data /per second/, or maybe two bits of data /per word/ for someone who continuously speaks for 70 years. Derate to a reasonable percentage of a lifetime spent generating output (10%? 1%?) and you're talking many, many bits per word. That's enough to Huffman compress their entire lifetime output with the "base model" entirely a dictionary, no need to even have a generative model!

jerf 3y ago
"Keep in mind that we're not trying to model the internal state of a human machine" Yes, that is exactly what we are discussing in this thread: Modelling the internal state of the human machine. The discussion of generating the exact stream of words is otherwise pointless. Fire lz at the collected output and call it a day; it isn't optimal but it's close enough. The entire conversation is whether we could recover the human consciousness from just the historical record.

csours 3y ago

> Hundreds of kilobytes is not a lot of information. But how much work is done by my mind when I read the output of the machine? It seems like humans are very strongly soft-wired (if not hard-wired, whatever that means) to try to understand other humans.