4 ms·
Another skill that'll be pretty valuable in this era is one many of the best technical leaders possess: the ability to coach desirable outcomes from squishy aut
by reb 4y ago
Another skill that'll be pretty valuable in this era is one many of the best technical leaders possess: the ability to coach desirable outcomes from squishy autonomous black boxes over time.
Getting excellent, exact performance out of deterministic systems is an impressive feat, but autonomy means variability, and getting excellent performance out of variable systems (especially illegible ones) is a different game.
- CSSer 4y agoI agree 100%. I first had this thought last night when reading “Eight Things to Know about LLMs” (select quote from section 8 below)[0] > Simply prompting a model to “think step by step” can lead it to perform well on entire categories of math and reasoning problems that it would otherwise fail on (Kojima et al., 2022). Similarly, even observing that an LLM consistently fails at some task is far from sufficient evidence that no other LLM can do that task (Bowman, 2022). The eerie thing to me is that this is coaching. I have never coached anything that isn’t alive by any definition. My feelings about this realization are ambivalent. [0]: https://cims.nyu.edu/~sbowman/eightthings.pdf https://cims.nyu.edu/~sbowman/eightthings.pdf
- kristov 4y agoDoesn't prompting "think step by step" simply increase the probability the model will generate its output in the style of places where it saw "step by step" before (eg: tutorial content)? It's not really "deciding" to think step by step. Maybe I am mistaken.