4 ms·

I find it fascinating that it seems like there's this emerging field of expertise around how to best interact with these gigantic LMs. I have no idea if this is

I find it fascinating that it seems like there's this emerging field of expertise around how to best interact with these gigantic LMs. I have no idea if this is like a passing fad during a gawky adolescent phase of the models, or if this is just a new thing that some people will always be at the cutting edge of. I gather that there's loose some precedent for the latter with the chess computer stuff: I've read that serious chess players heavily incorporate computers into their training and even that hybrid human/computer teams often outperform either/or teams. I'd love if someone who actually knows chess commented. Generating code via model sampling seems to have different, or at least exaggerated imperatives around "few-shot" tuning. One might wish to generate natural language for any number of purposes, but there is a probably a stronger "better"/"worse" gradient for code, and much like human language, excellent code is rare as a fraction of all code not only overall, but even by the same company or even by the same author. So you probably want the overall contours of like "this code compiles" from a big corpus, but to tune up the "this vectorizes well" eigen-tensor from Lemire's repo. Crazy times.

int_19h 4y ago
It reminds me of Vernor Vinge's "Rainbows End", where one of the basic classes they have in school is how to correctly word your search queries for best results. We might actually end up with a fancier version of that.
NhanH 4y ago
Human absolutely bring negative value to a chess match for at least 10 years now (when pairing with a computer). Even for Go AI, where it is only ~6 years old now, I'd still believe that human bring 0 value to a computer. 2-3 years ago, there was still some precise sequence where human might conceivable be dealing with (due to the nature of MCTS, human was better at reading deep narrow move sequences -- in go player term, human was sometimes better at life & death + some ladder sequences). Nowadays, the network are just way stronger than any human that we can't really follow what is going on in serious hour-long matches of the machine.
- qzw 4y ago
  > Human absolutely bring negative value… I have people skills; I am good at dealing with people. Can't you understand that? What the hell is wrong with you people?
  - benreesman 4y ago
    It sounds like somebody's got a case of the WFH movement is gutting middle management ;) In all seriousness though, this stuff is sobering to say the least. It's dangerous to extrapolate from 80% to 100%, but it's entirely possible that these things get good enough to take a meaningful bite out of software jobs. It would be supremely ironic if after a decade of thinking taxi and truck drivers were going to be put out of work it turns out computer programmers have their jobs threatened by generative models sooner!
    - int_19h 4y ago
      After looking at the outputs for anything reasonably complicated, well... If this steals our bread, then we deserve it.
      - qzw 4y ago
        Stackoverflow is probably just going to be filled with "What prompt do I need to write to get my AI to output the code I want?" questions instead.
    - qzw 4y ago
      It would be ironic indeed, but the whole thing with "taxi and truck drivers will soon be made obsolete by AI" was also based on extrapolating from 80% to 100% (IIRC, Uber was saying they'd be completely driverless by 2025).
- benreesman 4y ago
  Thanks for the informative reply! You sound quite knowledgable about this kind of thing so I'll pose a follow-up question(s). What game or broad category of game is the least dominated by machines these days? When I first saw the AlphaStar/StarCraft stuff it seemed that there was still plenty of room for humans to be competitive in a general sense, but that was some time ago. It's hard for me to imagine that 10 years from now there will be a game that isn't machine-dominated, but I could be wrong on either side: maybe that's true now, and maybe there's a set of games that still has DeepMind totally stumped!

int_19h 4y ago

It reminds me of Vernor Vinge's "Rainbows End", where one of the basic classes they have in school is how to correctly word your search queries for best results. We might actually end up with a fancier version of that.

NhanH 4y ago

Human absolutely bring negative value to a chess match for at least 10 years now (when pairing with a computer). Even for Go AI, where it is only ~6 years old now, I'd still believe that human bring 0 value to a computer. 2-3 years ago, there was still some precise sequence where human might conceivable be dealing with (due to the nature of MCTS, human was better at reading deep narrow move sequences -- in go player term, human was sometimes better at life & death + some ladder sequences). Nowadays, the network are just way stronger than any human that we can't really follow what is going on in serious hour-long matches of the machine.

qzw 4y ago
> Human absolutely bring negative value… I have people skills; I am good at dealing with people. Can't you understand that? What the hell is wrong with you people?
- benreesman 4y ago
  It sounds like somebody's got a case of the WFH movement is gutting middle management ;) In all seriousness though, this stuff is sobering to say the least. It's dangerous to extrapolate from 80% to 100%, but it's entirely possible that these things get good enough to take a meaningful bite out of software jobs. It would be supremely ironic if after a decade of thinking taxi and truck drivers were going to be put out of work it turns out computer programmers have their jobs threatened by generative models sooner!
  - int_19h 4y ago
    After looking at the outputs for anything reasonably complicated, well... If this steals our bread, then we deserve it.
    - qzw 4y ago
      Stackoverflow is probably just going to be filled with "What prompt do I need to write to get my AI to output the code I want?" questions instead.
  - qzw 4y ago
    It would be ironic indeed, but the whole thing with "taxi and truck drivers will soon be made obsolete by AI" was also based on extrapolating from 80% to 100% (IIRC, Uber was saying they'd be completely driverless by 2025).
benreesman 4y ago
Thanks for the informative reply! You sound quite knowledgable about this kind of thing so I'll pose a follow-up question(s). What game or broad category of game is the least dominated by machines these days? When I first saw the AlphaStar/StarCraft stuff it seemed that there was still plenty of room for humans to be competitive in a general sense, but that was some time ago. It's hard for me to imagine that 10 years from now there will be a game that isn't machine-dominated, but I could be wrong on either side: maybe that's true now, and maybe there's a set of games that still has DeepMind totally stumped!