3 ms·
I think also it's extremely hard to fairly judge high level decisions whose outcomes may not materialize for a very long time. This makes it genuinely hard for
by abeppu 1mo ago
I think also it's extremely hard to fairly judge high level decisions whose outcomes may not materialize for a very long time. This makes it genuinely hard for humans to get better (and one gets the impression that most of them don't) with more experience at the helm of a large and complex org. But it also means that it's hard to evaluate or train a machine on the task. You can maybe automate getting something plausible out, but how would you know if it was _good_ in a timely way? So how do you review a configuration or prompt change in this repo?
- psd1 1mo agoBy using historic data. You should strictly segregate training and testing data, to be sure that your model has internalised decision-making principles. [0] It is definitionally true that any given moment in history is unprecedented, and that old thinking is invalidated by paradigm shifts. But not every single variable is randomised. The launch of the PC didn't embarrass every executive, for example. 0: https://www.owlposting.com/p/an-ml-drug-discovery-startup-trying https://www.owlposting.com/p/an-ml-drug-discovery-startup-tr...
- sigbottle 1mo ago> You can maybe automate getting something plausible out, but how would you know if it was _good_ in a timely way? Contrary to the other guy, the answer is accountability, either personal, or higher-level systems like competition and ultimately Darwinism.