3 ms·

I think code monkeys or journalists etc. have GREAT experience with LLMs. In case of niche and more technical knowledge demanding task, LLM just fails. Eg. in

I think code monkeys or journalists etc. have GREAT experience with LLMs. In case of niche and more technical knowledge demanding task, LLM just fails. Eg. in my case, reverse engineering. It just spits BS all day long.

do_not_redeem 2y ago
Probably business types too. If you ask "what's the best email sending company" it will give you the most popular ones and all the right talking points to convince your underlings to switch. Now the salesfolk don't have to bother with golf games anymore.
- n4r9 2y ago
  A few months ago our product manager copypasted into a feature squad chat the results of asking an LLM to give some options for encoding route-navigation data for transmission from a server to a client device (both running our software). It gave about ten options ranging from half-decent but over-engineered (e.g. geoJSON) to bizarre (e.g. OSM, rinex, nmea). He didn't let on that it was an LLM response - the implication was that we should work through the list and anlayse the best one. But as both sides were using our own software so we could just use our own JSON encoding. Fortunately the head of engineering was also in the chat, immediately realised what was going on and dismissed the demand. Of course the real issue is that the PM is overreaching, but such behaviour is enabled and/or catalysed by LLMs. I don't doubt that similar things are happening in other companies that don't have the privilege of assertive leaders pushing back. The wasted effort could be significant.
skinner927 2y ago
Honestly if it’s not web (front and backend), it can’t figure it out. It makes sense though. Web has millions of examples. RE and other niche fields have far fewer examples, and the examples that do exist are extremely specific. How to pop a shell on device X: one example, extremely specific. How to write user auth for a web app: thousands of examples and generic enough it’ll work anywhere.

do_not_redeem 2y ago

Probably business types too. If you ask "what's the best email sending company" it will give you the most popular ones and all the right talking points to convince your underlings to switch. Now the salesfolk don't have to bother with golf games anymore.

n4r9 2y ago
A few months ago our product manager copypasted into a feature squad chat the results of asking an LLM to give some options for encoding route-navigation data for transmission from a server to a client device (both running our software). It gave about ten options ranging from half-decent but over-engineered (e.g. geoJSON) to bizarre (e.g. OSM, rinex, nmea). He didn't let on that it was an LLM response - the implication was that we should work through the list and anlayse the best one. But as both sides were using our own software so we could just use our own JSON encoding. Fortunately the head of engineering was also in the chat, immediately realised what was going on and dismissed the demand. Of course the real issue is that the PM is overreaching, but such behaviour is enabled and/or catalysed by LLMs. I don't doubt that similar things are happening in other companies that don't have the privilege of assertive leaders pushing back. The wasted effort could be significant.

skinner927 2y ago

Honestly if it’s not web (front and backend), it can’t figure it out. It makes sense though. Web has millions of examples. RE and other niche fields have far fewer examples, and the examples that do exist are extremely specific. How to pop a shell on device X: one example, extremely specific. How to write user auth for a web app: thousands of examples and generic enough it’ll work anywhere.