3 ms·

This dovetails with something I've been thinking lately; the foundation of civilization is standing on the shoulders of giants (ie, I might make small contribut

This dovetails with something I've been thinking lately; the foundation of civilization is standing on the shoulders of giants (ie, I might make small contributions but I didn't invent compilers, CPUs, mining, agriculture, etc). Progress in large part is figuring out better ways of doing that (language, written language, printing press, internet access, etc). When you look at things that way - LLMs start to seem deeply ground-breaking (assuming we work out the confabulation kinks). EDIT: fixed grammar/typos. Maybe I should have had an LLM proof-read this...

visarga 2y ago
> the foundation of civilization is standing on the shoulders of giants. Progress in large part is figuring out better way of doing that It's so much easier to imitate than to invent something truly novel and useful. Let's do a bit of napkin math. A human lifetime is about 500M words. GPT-4 used up about 30,000 human lifetimes of language. But cultural evolution took 200K years and 120B people to get here, about 4 million times the size of GPT-4's training set. That is of course hand wavy, but it shows progress is a million times harder than imitation. We really are standing on the shoulders of giants, or a very long chain of people. If we forgot all the knowledge preserved by language it would take us the same effort to recover as the first time around. I think all progress comes from search - search for experience (data), and search for understanding (data compression). This feeds on itself, but is coupled with the search space - the real world. And the world is not eager to tell us all its secrets. AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well.
- joshmarlow 2y ago
  > AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well. I think Kevin Kelly made a similar point in explaining his skepticism around intelligence explosions; even if we build something much much smarter than us, the thing will still need to do experiments to fine-tune it's (super-human nuanced) understanding of physics/biology/whatever - and the clock-cycle of external reality isn't speeding up like our computation is. I think something smarter than us could design better/more informative experiments than we could to gather information about the world. That being said, I think his/your point is insightful.
- hackable_sand 2y ago
  Well put.
Terr_ 2y ago
> the foundation of civilization is standing on the shoulders of giants [...] Progress in large part is figuring out better ways of doing that Cynical take: When it comes to helping someone find exactly the right piece of esoteric knowledge needed... There's no profit for that in a search engine, and an LLM reflects word associations rather than facts. IANAScienceHistorian, but I find myself thinking of how DNA analysis would be different if nobody had found Thermus aquaticus, with it's extra-hardy variant of polymerase, or how the history of stealth aircraft was kicked off when someone realized the implications of Petr Ufimtsev's equations for reflected EM waves. (A work which went--heh--under the radar inside the USSR.)
bryanrasmussen 2y ago
>When you look at things that way - LLMs start to seem deeply ground-breaking (assuming we work out the confabulation kinks). even if you don't work out the confabulation kinks, given what was said by the first post, it still seems groundbreaking enough. Although I can't figure out why every time I do anything with LLMs it's not worth it, I guess because I am using it for things I am an expert in, it doesn't help. Actually maybe it's like the article Suggestions from Idiots - https://medium.com/luminasticity/suggestions-from-idiots-6b023acc30ae https://medium.com/luminasticity/suggestions-from-idiots-6b0... - suggesting distracting instead of diverting is not helpful when diverting is the better word in context, but for someone who doesn't know what word to use in the context or uses diverting when it is not the best word the suggestion is useful. If that's the case what he got out of Claude is probably not that great, but it is passable, just like the word distraction instead of diverting in the right context is still passable, or when I get back a time conversion function from CoPilot that fits all but a few edge cases, it's just fine as long as the edge case never hits it, then it sucks - maybe there's some edge cases where his LED diffusers won't work quite as well as they could if written by an expert. Then again it might be that since what he is doing is analog edge cases and tolerance for failure is such that when it does fail it is not as problematic as when a bit of code fails because the computer dealing with the output is not as forgiving as the human eye and brain.

visarga 2y ago

> the foundation of civilization is standing on the shoulders of giants. Progress in large part is figuring out better way of doing that It's so much easier to imitate than to invent something truly novel and useful. Let's do a bit of napkin math. A human lifetime is about 500M words. GPT-4 used up about 30,000 human lifetimes of language. But cultural evolution took 200K years and 120B people to get here, about 4 million times the size of GPT-4's training set. That is of course hand wavy, but it shows progress is a million times harder than imitation. We really are standing on the shoulders of giants, or a very long chain of people. If we forgot all the knowledge preserved by language it would take us the same effort to recover as the first time around. I think all progress comes from search - search for experience (data), and search for understanding (data compression). This feeds on itself, but is coupled with the search space - the real world. And the world is not eager to tell us all its secrets. AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well.

joshmarlow 2y ago
> AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well. I think Kevin Kelly made a similar point in explaining his skepticism around intelligence explosions; even if we build something much much smarter than us, the thing will still need to do experiments to fine-tune it's (super-human nuanced) understanding of physics/biology/whatever - and the clock-cycle of external reality isn't speeding up like our computation is. I think something smarter than us could design better/more informative experiments than we could to gather information about the world. That being said, I think his/your point is insightful.
hackable_sand 2y ago
Well put.

Terr_ 2y ago

> the foundation of civilization is standing on the shoulders of giants [...] Progress in large part is figuring out better ways of doing that Cynical take: When it comes to helping someone find exactly the right piece of esoteric knowledge needed... There's no profit for that in a search engine, and an LLM reflects word associations rather than facts. IANAScienceHistorian, but I find myself thinking of how DNA analysis would be different if nobody had found Thermus aquaticus, with it's extra-hardy variant of polymerase, or how the history of stealth aircraft was kicked off when someone realized the implications of Petr Ufimtsev's equations for reflected EM waves. (A work which went--heh--under the radar inside the USSR.)

bryanrasmussen 2y ago

>When you look at things that way - LLMs start to seem deeply ground-breaking (assuming we work out the confabulation kinks). even if you don't work out the confabulation kinks, given what was said by the first post, it still seems groundbreaking enough. Although I can't figure out why every time I do anything with LLMs it's not worth it, I guess because I am using it for things I am an expert in, it doesn't help. Actually maybe it's like the article Suggestions from Idiots - https://medium.com/luminasticity/suggestions-from-idiots-6b023acc30ae https://medium.com/luminasticity/suggestions-from-idiots-6b0... - suggesting distracting instead of diverting is not helpful when diverting is the better word in context, but for someone who doesn't know what word to use in the context or uses diverting when it is not the best word the suggestion is useful. If that's the case what he got out of Claude is probably not that great, but it is passable, just like the word distraction instead of diverting in the right context is still passable, or when I get back a time conversion function from CoPilot that fits all but a few edge cases, it's just fine as long as the edge case never hits it, then it sucks - maybe there's some edge cases where his LED diffusers won't work quite as well as they could if written by an expert. Then again it might be that since what he is doing is analog edge cases and tolerance for failure is such that when it does fail it is not as problematic as when a bit of code fails because the computer dealing with the output is not as forgiving as the human eye and brain.