3 ms·

“Radiologists do far more than study images. They advise other doctors and surgeons, talk to patients, write reports and analyze medical records. After identify

by sulam 1y ago

“Radiologists do far more than study images. They advise other doctors and surgeons, talk to patients, write reports and analyze medical records. After identifying a suspect cluster of tissue in an organ, they interpret what it might mean for an individual patient with a particular medical history, tapping years of experience.” Now think about how much of software development is typing out the code vs talking to people, getting a clear definition of the problem, debugging, etc. (I would love an LLM that could debug problems in production — but all they can do is tell me stuff I already know). Then layer on that there are far more ideas for what should be built than you have time to actually build in every organization I’ve ever worked in. I’m not worried about my job. I’m more worried my coworkers won’t realize what a great tool this is and my company will be left in the dust.

deleted 1y ago
[deleted]
agos 1y ago
that great tool does not need ten years of experience to use. your coworkers will be able to catch up quite easily
- sulam 1y ago
  Fwiw I think this is not true right now. It might be true in the future, I guess. Today you have to learn some pretty specific techniques and they don’t come naturally to most engineers I think. They are accustomed to one-shot Google searches, not prompt engineering. Yes, you can learn how to prompt engineer, but it takes trial and error. A lot of my co-workers today still believe the LLM is useless because it hallucinates API definitions. They very much are not used to tools that require the level of hand-holding you need today. Either teaching them is going to get easier, which is somewhat plausible actually — or you think LLMs are going to get dramatically easier to prompt with more reliable output. I actually think that’s somewhat implausible. Log(compute) is a pretty hard number to move past a certain point and it seems we’ve hit that point already. RLHF is great, but it doesn’t scale well and the result is that some tools are hard to build. Giving me millions of context tokens is neat, but if it’s wildly expensive and confuses the model on top of that, it’s not actually useful. I do think there is room to improve and of course there are new techniques to be discovered. The models may even help discover some of them, although I am skeptical that they will find Everest by hill climbing when they’re currently on Pike’s Peak. Or maybe we really need to find Olympus Mons. Tl;dr — maybe!

by sulam 1y ago

deleted 1y ago

[deleted]

agos 1y ago

that great tool does not need ten years of experience to use. your coworkers will be able to catch up quite easily

sulam 1y ago
Fwiw I think this is not true right now. It might be true in the future, I guess. Today you have to learn some pretty specific techniques and they don’t come naturally to most engineers I think. They are accustomed to one-shot Google searches, not prompt engineering. Yes, you can learn how to prompt engineer, but it takes trial and error. A lot of my co-workers today still believe the LLM is useless because it hallucinates API definitions. They very much are not used to tools that require the level of hand-holding you need today. Either teaching them is going to get easier, which is somewhat plausible actually — or you think LLMs are going to get dramatically easier to prompt with more reliable output. I actually think that’s somewhat implausible. Log(compute) is a pretty hard number to move past a certain point and it seems we’ve hit that point already. RLHF is great, but it doesn’t scale well and the result is that some tools are hard to build. Giving me millions of context tokens is neat, but if it’s wildly expensive and confuses the model on top of that, it’s not actually useful. I do think there is room to improve and of course there are new techniques to be discovered. The models may even help discover some of them, although I am skeptical that they will find Everest by hill climbing when they’re currently on Pike’s Peak. Or maybe we really need to find Olympus Mons. Tl;dr — maybe!