4 ms·

> There is certainly no way to recover Marcus Aurelius just from his writing no matter how much fancy AI is thrown at it; it is simply mathematically impossible

by addaon 3y ago

> There is certainly no way to recover Marcus Aurelius just from his writing no matter how much fancy AI is thrown at it; it is simply mathematically impossible. Without addressing whether I agree with your conclusion or not, I disagree that this is a rigorous argument. The question isn't how much complexity the text processing machine that was the Original Marcus has, it's how much of that complexity was unique to OM. We have a large amount of information about modern humans, and a good chunk about historical humans. You would need to believe not just that OM requires many times more information than the hundreds of kilobytes we have, but that the OM delta patch to standard human or period-adjusted human /also/ has more information that the hundreds of kilobytes we have. I'm not sure it's obvious that humans are that interesting or that different, even interesting humans.

jerf 3y ago
Hundreds of kilobytes is not a lot of information. Even guessing gigabytes would be going very controversially low. Just the delta versus some "average" human genome would easily be into the hundreds kilobytes. Even just modeling a human as the sum total of their sensory input, itself a simplification transparently obviously insufficient, hundreds of kilobytes covers just a few seconds of their existence, or with compression, minutes or maybe hours, tops. Imagine trying to create a Marcus Aurelius that correctly output exactly the contents of his writing, while also being acceptably a human analog. If you think you can stick that into a few hundred kilobytes, there's a 500,000 Euro prize you should be starting forthwith as you're claiming many orders of magnitude improvement over the current best: http://prize.hutter1.net/ http://prize.hutter1.net/
- addaon 3y ago
  > Hundreds of kilobytes is not a lot of information. Humans are very, very predictable. It's not clear that the deviation of one human from the mean has a lot of information. It's not clear that it /doesn't/, either -- but I'd sure as heck take the under on gigabytes for novelty of text output. Keep in mind that we're not trying to model the internal state of a human machine; instead, we're discussing how much additional information is needed to align the output of a generic human text generator with the output of a specific individual. A lifetime is about two billion seconds -- so a gigabyte is four bits of data /per second/, or maybe two bits of data /per word/ for someone who continuously speaks for 70 years. Derate to a reasonable percentage of a lifetime spent generating output (10%? 1%?) and you're talking many, many bits per word. That's enough to Huffman compress their entire lifetime output with the "base model" entirely a dictionary, no need to even have a generative model!
  - jerf 3y ago
    "Keep in mind that we're not trying to model the internal state of a human machine" Yes, that is exactly what we are discussing in this thread: Modelling the internal state of the human machine. The discussion of generating the exact stream of words is otherwise pointless. Fire lz at the collected output and call it a day; it isn't optimal but it's close enough. The entire conversation is whether we could recover the human consciousness from just the historical record.
- csours 3y ago
  > Hundreds of kilobytes is not a lot of information. But how much work is done by my mind when I read the output of the machine? It seems like humans are very strongly soft-wired (if not hard-wired, whatever that means) to try to understand other humans.

by addaon 3y ago

jerf 3y ago

Hundreds of kilobytes is not a lot of information. Even guessing gigabytes would be going very controversially low. Just the delta versus some "average" human genome would easily be into the hundreds kilobytes. Even just modeling a human as the sum total of their sensory input, itself a simplification transparently obviously insufficient, hundreds of kilobytes covers just a few seconds of their existence, or with compression, minutes or maybe hours, tops. Imagine trying to create a Marcus Aurelius that correctly output exactly the contents of his writing, while also being acceptably a human analog. If you think you can stick that into a few hundred kilobytes, there's a 500,000 Euro prize you should be starting forthwith as you're claiming many orders of magnitude improvement over the current best: http://prize.hutter1.net/ http://prize.hutter1.net/

addaon 3y ago
> Hundreds of kilobytes is not a lot of information. Humans are very, very predictable. It's not clear that the deviation of one human from the mean has a lot of information. It's not clear that it /doesn't/, either -- but I'd sure as heck take the under on gigabytes for novelty of text output. Keep in mind that we're not trying to model the internal state of a human machine; instead, we're discussing how much additional information is needed to align the output of a generic human text generator with the output of a specific individual. A lifetime is about two billion seconds -- so a gigabyte is four bits of data /per second/, or maybe two bits of data /per word/ for someone who continuously speaks for 70 years. Derate to a reasonable percentage of a lifetime spent generating output (10%? 1%?) and you're talking many, many bits per word. That's enough to Huffman compress their entire lifetime output with the "base model" entirely a dictionary, no need to even have a generative model!
- jerf 3y ago
  "Keep in mind that we're not trying to model the internal state of a human machine" Yes, that is exactly what we are discussing in this thread: Modelling the internal state of the human machine. The discussion of generating the exact stream of words is otherwise pointless. Fire lz at the collected output and call it a day; it isn't optimal but it's close enough. The entire conversation is whether we could recover the human consciousness from just the historical record.
csours 3y ago
> Hundreds of kilobytes is not a lot of information. But how much work is done by my mind when I read the output of the machine? It seems like humans are very strongly soft-wired (if not hard-wired, whatever that means) to try to understand other humans.