3 ms·

I mean paying several hundred to thousands of grad students to RLHF for several years and you get a corpus of grad-student text. I'm not surprised at all. AI co

by meroes 2y ago

I mean paying several hundred to thousands of grad students to RLHF for several years and you get a corpus of grad-student text. I'm not surprised at all. AI companies hire grad students to RLHF in every subject matter (chemistry, physics, math, etc). The grad-students write the prompts, correct the model, and all of that is fed into a "more advanced" model. It's corpi of text. Repeat this for every grade level and subject. Ask the model that's being trained on chemistry grad level work a simple math question and it will probably get it wrong. They aren't "smart". It's aggregations of text and ways to sample and then predict.

fnordpiglet 2y ago
Except you’re talking about a general purpose foundation model that’s doing all these subjects at once. It’s not like you choose the subject specific model with Claude or gpt-01. The key isn’t whether these things are smart or not. The key is that they put something that can answer basic grad level questions on almost any subject. For people that don’t have a graduate level education in any subject this is a remarkable tool. I don’t know why the statement that “wow this is useful and a remarkable step forward” is always met with “yeah but it’s not actually smart.” So? Half of all humans have an IQ less than 100. They’re not smart either. Is this their value? For a machine, being able to produce accurate answers to most basic graduate level questions is -science fiction- regardless of whether it’s “smart.” The NLP feat alone is stunning, and going from basically one step above gibberish to “basic grad school” in two years is a mouth dropping rate of change. I suspect folks who quibble over whether it’s “real intelligence” or simply a stochastic parrot have lost the ability to dream.
- meroes 2y ago
  Well ya once each project, e.g. “grad level math”, “k—12 math”, “undergrad math”, “k-12 chemistry”, etc is sufficient they are all fed into a larger more powerful model. Maybe my RLHF work does make it harder for me to dream, but I teach models math which means a lot of prompt writing, and yet I have not found a way to have the model teach me math I don’t know yet (and there’s a lot I don’t know). It’s fun to play around with, but I still gravitate toward the isolated texts, not the aggregation as too much is lost or averaged in my opinion/experience. But hey maybe I’m overtrained on the traditional learning methods.

by meroes 2y ago

fnordpiglet 2y ago

Except you’re talking about a general purpose foundation model that’s doing all these subjects at once. It’s not like you choose the subject specific model with Claude or gpt-01. The key isn’t whether these things are smart or not. The key is that they put something that can answer basic grad level questions on almost any subject. For people that don’t have a graduate level education in any subject this is a remarkable tool. I don’t know why the statement that “wow this is useful and a remarkable step forward” is always met with “yeah but it’s not actually smart.” So? Half of all humans have an IQ less than 100. They’re not smart either. Is this their value? For a machine, being able to produce accurate answers to most basic graduate level questions is -science fiction- regardless of whether it’s “smart.” The NLP feat alone is stunning, and going from basically one step above gibberish to “basic grad school” in two years is a mouth dropping rate of change. I suspect folks who quibble over whether it’s “real intelligence” or simply a stochastic parrot have lost the ability to dream.

meroes 2y ago
Well ya once each project, e.g. “grad level math”, “k—12 math”, “undergrad math”, “k-12 chemistry”, etc is sufficient they are all fed into a larger more powerful model. Maybe my RLHF work does make it harder for me to dream, but I teach models math which means a lot of prompt writing, and yet I have not found a way to have the model teach me math I don’t know yet (and there’s a lot I don’t know). It’s fun to play around with, but I still gravitate toward the isolated texts, not the aggregation as too much is lost or averaged in my opinion/experience. But hey maybe I’m overtrained on the traditional learning methods.