7 ms·
I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as w
by benreesman 4y ago
I’m well aware that this is an unpopular opinion.
When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet.
People read too much into this.
The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sentence-was-on-the-internet).
- benreesman 4y agoThe whole show in ML these days is in the infra around massively distributed, redundant, tunable, train the same fucking transformer we’ve all been using since 2017.
- rytill 4y agoWhat about diffusion models? (And all the other things with less publicity that aren’t that)
- benreesman 4y agoSure, I’m obviously being a big glib. So it’s transformers plus let’s force some loss into the super-resolution pipeline. “Residual” sounds a lot fancier than X = fn + X, and “Diffusion” sounds a lot fancier than “let’s do deconv repeatedly and jam a loss term in there”. My point is all of this stuff is on Huggingface. You/I/We just don’t have 5k A100s for 6 months, so we can’t play.
- tehsauce 4y agoIf you can find a more efficient or effective approach than this “redundant” one you’ll be rich. There’s a lot of smart people working in this space.
- benreesman 4y agoI’m not the person to break this logjam of “moar GPUs”. But I agree that someone will, and I agree that person will do quite well.
- MontyCarloHall 4y agoI completely agree. Nonetheless, this approach of taking a fairly complex writing prompt and simply regurgitating contextually similar text from the internet often yields 100% meaningful results. This suggests that our intelligence is a lot more mimetic than we'd like to believe, which is perhaps an even more unpopular opinion.
- benreesman 4y agoI tend to agree that the math gets a lot simpler if the sun doesn’t rotate around a mystical axiom that humans are “special”.
- benreesman 4y agoHigh-energy physicists are about as serious as it gets, but they’re still dragging their feet around many worlds because it has uncomfortable implications for consciousness and free will. Homo Sapiens has a serious agenda around being something other than the smartest chimpanzee.
- alar44 4y agoNo, they're "dragging their feet" because it's impossible to test, it's unfalsifiable.
- benreesman 4y agoSuperposition collapse is impossible to test by definition of not including the observer in the system. If we just admit the possibility that the math was right all along we win, the math works, and we lose “I am a unique, distinct entity.” I’m on team math.
- IdealeZahlen 4y agoThat's math;not physics. You need experiments in physics.
- tehsauce 4y agoWorth noting that in the limit the distribution P(human did/said A | context B) is a complete definition of human behavior. If you could model this perfectly, that would be a perfect model of a human ie identical.
- benreesman 4y agoExactly right. But extremely serious scientists, very smart people, are still drawing epicycles on blackboards studying “consciousness”.
- legalcorrection 4y agoPretending consciousness doesn't exist or that it has no function I think reflects poorly on someone who studies human behavior.
- benreesman 4y agoI’m saying that the idea of a bright line between the emergent behavior of a dolphin and a human is very pre-Copernicus. Studying, even measuring the capabilities of an animal is science. Justifying a soul is the purview of spirituality, not science. (Nothing against spirituality, I have a spiritual life, I just don’t confuse it with science).
- legalcorrection 4y agoI strongly disagree. Pure behaviorism is just willful blindness. Consciousness is a real phenomenon, as any person not philosophically committed to denying its existence can tell you. It's front and center of our experience of human cognition. It would be quite strange for it to serve no function in the human mind. Yeah, it's hard to quantify and isolate and experiment on, but that just speaks to either current limitations of human science, or possibly to limitations that cannot be surpassed. Given how much mileage certain philosophical movements have gotten out the common intuition that emerged during the Elightenment that everything is scientifically tractable, I understand the resistance to accepting these limitations and opening the door to all of the philosophical consequences of that intuition failing. But sorry, reality doesn't care about your philosophical attachments.
- NavinF 4y ago> I’m well aware that this is an unpopular opinion It’s so unpopular that literally every post on HN/proggit related to language models has the same comment ;)
- benreesman 4y agoHaha well played. I’ll counter your non-sequitor with an appeal to authority ;) I used to do this for a living.
- NavinF 4y agoIf you’re a language model, congrats on hitting 0.00 perplexity on the HN comments dataset. Your sponsors would be proud :P
- benreesman 4y agoIf I was going to hook a big language model to HN I wouldn’t do it under my real name. But… I take your meaning. I think I was subconsciously channeling OPT because that’s what the thread is about. I agree it’s the kind of thing a fine-tuned HN chat bot would say all the friggin time.
- bilsbie 4y ago> but it’s still just maximizing P(this-sentence-was-on-the-internet). But are we doing any different?
- benreesman 4y agoNot even a little!
- bitL 4y agoIt seems like the bigger transformer you have, the less the generated text feels like a speech of somebody with Wernicke's encepalopathy and more like a normal person, but there is still the feeling of some "brain lesion" in there somewhere.
- Veedrac 4y ago1. They aren't overfit. 2. It was far from obvious a priori that this could work at all. The existence of the sentiment ‘lol you trained a big model on X so obviously it produces a good probability distribution on X’ only exists because big models proved to be extraordinarily more effective than anyone expected.
- benreesman 4y agoWhen you’re churning through terrawords of training data and aren’t cross-validating against proportionally-sized hold out sets? It’s not a certainty that you’re overfitting but the burden is to show otherwise. Besides, if your corpus is asymptomatically everything, why wouldn’t fitting it perfectly fitting it be the goal? It would probably be more accurate to say that the bias/variance tradeoff loses meaning as the training set goes to infinity. Grapevine is that the big actors are going to 5x their flops at any price in the next three years. What does validation loss even mean at that scale?
- Veedrac 4y ago> Besides, if your corpus is asymptomatically everything, why wouldn’t fitting it perfectly fitting it be the goal? That's called Google, and it serves a different purpose. Perfectly learning the distributional properties of text instead is not overfitting, that's just fitting.
- trention 4y ago>The existence of the sentiment ‘lol you trained a big model on X so obviously it produces a good probability distribution on X’ only exists because big models proved to be extraordinarily more effective than anyone expected. No, the "popularity" of that sentiment exists because of their "effectiveness". That sentiment was existing and being voiced 10 years ago.
- benreesman 4y agoYou’ve stated this point better than I did. Otherwise brilliant and rational people just go all mystical when we start talking about meaningless words like “intelligence” or “consciousness”. An animal or a human or a big matrix perform X well on Y task. That’s quantitative and objective. All this “is it smart” bullshit is thinly-veiled “how does the world still rotate around my subjective experience given this thing writes better Tolkien fan fiction than I do”. Performance on tasks. Everyone else shuffle over to the Philosophy department.
- deleted 4y ago[deleted]
- the8472 4y agoPart of the evaluation procedures for the models like these is checking whether the output they produce is present in the training set. The fraction of samples being verbatim reurgitations is low unless you prompt-engineer your way to that, copilot being able to recite the fast inverse square root function being a common example. But the ability to recite some samples is not proof of inability to create novel outputs.
- benreesman 4y agoOh sure. I didn’t mean to imply that GPT-6 will have memorized Wikipedia. As you start to get to the size and comprehensiveness in the corpus that’s going on now, novel model outputs approach being something that might not have been in the training set, but likely will be in the future.
- trention 4y ago>The fraction of samples being verbatim reurgitations is low Copilot spitting out a function out of its training data with changed variable names to match those in your file(s) - and no one is actually testing what proportions of results those are - is still regurgitation.
- the8472 4y ago> and no one is actually testing what proportions of results those are The idea that models could possibly overfit the training data is hardly a new idea. It's standard practice to test for that. Check section 7 of the PaLM paper for example. https://arxiv.org/abs/2204.02311 https://arxiv.org/abs/2204.02311
- benreesman 4y agoCopilot is weird. The N in the NLP training set and arch implies a fuzzy match. Why anyone would rather start with a plausible but broken-in-a-subtle way buffer rather than a blank one is beyond me.
- deleted 4y ago[deleted]
- leobg 4y agoThere’s a case to be made for the proposition that diffusion is what makes the kind of “intelligence” that we value. If you cook a great dish and you cannot explain how you did it, you get a lot of credit for being a great cook. If, on the other hand, you can point to exactly one recipe that you followed, you will get a lot less credit. So in a sense, the harder it is to trace down the inspirations from which a composition sprung, the higher we value it.
- telotortium 4y agoYou better watch what you're saying - soon the champion (in the medieval sense) of ML, Gwern, will come to defend his liege's honor. I see the other champion, Veedrac, has already come out :)
- benreesman 4y agoOh I took a licking from @gwern (who I do respect but isn’t as insightful as his fame would imply) a few weeks ago for denying that the Sigularity Is Near. Being Mr Who Ever Heard of This Guy myself, I took it as a compliment. :) @veedrac on the other hand, who is smart, pretty clearly got the worse on that friendly little skirmish.