4 ms·
You’re comparing an LLM to expert programmers. Compare an LLM on the same task versus the average college student. And try it for a math problem. A poetry pro
by kenjackson 2y ago
You’re comparing an LLM to expert programmers. Compare an LLM on the same task versus the average college student. And try it for a math problem. A poetry problem. Ask it a more complex question about history or to do an analysis of an essay you wrote.
Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose?
- aleph_minus_one 2y ago> Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose? You wouldn't randomly selected an arbitrary adult from the USA to do a brain surgery on you, so this argument is rabulistic.
- kenjackson 2y agoBrain surgery requires a license. But I do expect an arbitrary adult to be able to follow instructions. Ok. How about you give me a text based task where you would pick the random adult over the LLM?
- aleph_minus_one 2y ago> Brain surgery requires a license. This is rather a red-tape problem. :-)
- nyclounge 2y agoI think you and the parent may be talking about 2 different things. Do I want to use an LLM to do it from business owner perspective? Yeah probably it is cheaper and more convenient. Which one I want to use, depending the problem we are solving here right? I'm more concern about the integrity of the current digital infrastructure. In that sense I would NOT trust ANY thing really important to anything digital, much less to LLM. Can I use it for exploration then require an actually human expert approval/edit. Absolutely! As long as the digital doesn't result in significant physical or financial damage. Edit: and for HN ppl, of course the LLM will have have to be open weight and all and run locally in a air gaped GPU, preferably in a Faraday cage.
- daveguy 2y agoI would chose a random person from my company that was hired to work in that domain to solve problems in that domain. Yes, regardless of the position. Accountant in the domain, yes. Office organizer in the domain, yes. Essentially anyone in the domain, yes. No offense, but by restricting the selection to the general human population you're setting a low bar for LLMs here.
- kenjackson 2y agoIf the bar is for LLMs to replace domain experts about four years after introduction then yes, they are failing miserably. But if you were to go back to 2020 and ask if your take a random human over a the state of the art AI to answer a text question you’d take the random human every time except for arithmetic (and you’d have to write it in math notation and not plain English). And if you were to ask AI experts when would you chose an AI they’d say at least not for a decade or two, if ever.
- daveguy 2y agoI wasn't talking about how impressive AI systems are, or how far they've come. I was talking about the fact that any random human with any experience in a specific field -- even though they are not a domain expert -- is going to do better than an LLM. Or, human common sense >>>> what LLMs are doing.
- kenjackson 2y agoWe will have to agree to disagree about your fundamental point.
- daveguy 2y agoFair enough. We will see.