3 ms·
Is that the objective? Is there actually even an objective at this point? I don't know anyone in the field that expected training an LLM to autocomplete senten
by kromem 3y ago
Is that the objective? Is there actually even an objective at this point?
I don't know anyone in the field that expected training an LLM to autocomplete sentences would have led to being able to explain why a joke it had never seen before was funny.
It seems like objectives have gone out the window and we're now at a point where billion dollar companies are frantically throwing everything at the wall to see what sticks before their competitors do, with what sticks frequently being counterintuitive to prior predictions or out of scope from initial intentions.
If a LLM can cure cancer by being able to analyze connections between advances across the last decade of published research more successfully than human brains limited by the rule of seven plus or minus two, would a continued failure to conform to prompt specifications still effectively be a failure?
Maybe early computer scientists overemphasized the role of cold logic in intelligence and there's greater strides in the epiphany of a shower thought or Feynman's "thinking about twelve problems concurrently" with the next leaps in progress involving further distancing models from constraints.
I think we'd be wise to broadly toss out everything we thought we'd know about AI as a philosophical domain based on past theorizing and consider what's emerged over the past 36 months with fresh eyes and consideration.
And I don't see greater determinism or perfect bounds as being where this goes, particularly given the potential hardware shift to optoelectronic neural networks where arguably going all in on black boxes and stochastic results has actual physical advantages.