6 ms·
If you spent even more time with GPT-4 it would be evident that it is definitely not. Especially if you try to use it as some kind of autonomous agent.
by akiselev 3y ago
If you spent even more time with GPT-4 it would be evident that it is definitely not. Especially if you try to use it as some kind of autonomous agent.
- p1esk 3y agoWhat have you tried to do with it?
- akiselev 3y agoUse it to analyze the California & US Code, the California & Federal Codes of Regulation, and bills currently in the California legislation & Congress. It's far from useless but far more useful for creative writing than any kind of understanding or instruction following when it comes to complex topics. Even performing a map-reduce over large documents to summarize or analyze them for a specific audience is largely beyond it. A 32K context size is a pittance when it comes to a single Title in the USC or CFR, which average into the millions of tokens each.
- p1esk 3y agoInteresting - do you believe average humans (not professional lawyers) would do better on this task?
- karpierz 3y agoProbably not. I'm not sure that pointing out that LLMs are as useful for parsing legal code as the average human is something to brag about though.
- p1esk 3y agoYou're not sure that having an AI which is (at least) as intelligent as an average human is something to brag about? Seriously?
- karpierz 3y agoWhere did you get that the LLM was as _intelligent_ as a human? All we've shown is that LLMs are as useful for parsing legal text as the average human. Which is to say, not. A dog is also as useful for parsing legal texts as the average human. So is a rock.
- p1esk 3y agoWhere did you get that the LLM was as _intelligent_ as a human? First hand experience -I’ve been using it daily for the past two months.
- karpierz 3y agoAh, if that's what you mean then there are plenty of intelligent systems out there. I've used Google search for decades and it's been able to answer questions better than humans ever could. Same for Google Maps, though arguably they're the same system at this point. My calculator is far more intelligent than any human I've met, at least when it comes to adding large numbers. My compiler can detect even the slightest syntax error with impeccable accuracy. Microsoft word has an incredible vocabulary. Wikipedia knows more historical events than any human dead or alive. And so on. Shit, users thought Eliza was intelligent in the 60s. If what you really mean is that LLMs are cool and useful, then sure. Just say that instead of couching it in some vague claim of intelligence.
- p1esk 3y agoNo, what I meant was GPT-4 is more intelligent than most humans I interact with on a daily basis. In the fullest meaning of that word.
- karpierz 3y agoThere are a lot of different ways to interpret the word intelligent, so let me rephrase: When you say "intelligent", what do you mean exactly? What might help is describing what specific interactions give you the impression that GPT-4 is intelligent?
- akiselev 3y agoYes. I can parse them just fine after reading a single book called Introduction to Legal Reasoning [1]. I can also autonomously take notes and keep track of a large context using a combination of short and long term memory despite not having any kind of degree let alone experience or a license to practice law. How do you think people become lawyers and how smart do you think the average lawyer actually is? The problem is that there's hundreds of thousands if not millions of pages, not that it requires superhuman intelligence to understand. Even if it were capable of intelligence in the bottom quartile of humanity it would be SO MUCH more useful than it is now because I'd be able run and get something useful out of thousands of models in parallel. As it stands now GPT4 fails miserably at scaling up the kind of reasoning and understanding that even relatively stupid humans are capable of. [1] https://www.amazon.com/Introduction-Legal-Reasoning-Edward-Levi/dp/022608972X https://www.amazon.com/Introduction-Legal-Reasoning-Edward-L...
- streakfix 3y agoDid you try fine tuning gpt4 with that book as input?
- akiselev 3y agoOpenAI doesn't support fine tuning of GPT4 and with context stuffing,the more of the book I include in the input the less of the bills I can include - which, again, are millions of tokens - and the less space there is for memory.
- p1esk 3y agoI believe you. But at the same time they showed during the demo how it can do taxes, using a multi page document. An ability to process longer documents seems more like an engineering challenge rather than a fundamental limitation.
- akiselev 3y agoDoing taxes using a few small forms designed together by the same agency is not as impressive as you think it is. The instructions are literally printed on the form in English for the kind of people who you consider dumber than ChatGPT. It quickly breaks down even at 8k with legislation that is even remotely nontrivial.
- hackernewds 3y agoIf you spent even more time with GPT-4 it would be evident that it definitely is. Especialy if you try to use it as some kind of autonomous agent. (Notice how baseless comments can sway either way)
- taneq 3y ago> (Notice how baseless comments can sway either way) No they can’t! ;)
- 8bitsrule 3y agoLet's let John Cleese decide. Or maybe someone was looking for Abuse!
- callalex 3y agoEngaging with this is probably a mistake, but remember the burden of proof is on the claimant. What examples do you have of ChatGPT for example, learning in a basic classroom setting, or navigating an escape room, or being inspired to create its own spontaneous art, or founding a startup, or…
- derefr 3y agoAI research has put hardly any effort into building goal-directed agents / A-Life since the advent of Machine Learning. A-Life was last really "looked into" in the '70s, back when "AI" meant Expert Systems and Behavior Trees. All the effort in AI research since the advent of Machine Learning, has been focused on making systems that — in neurological terms — are given a sensory stimulus of a question, and then passively "dream" a response to said question as a kind of autonomic "mind wandering" process. (And not even dynamic systems — these models always reach equilibrium with some answer and effectively halt, rather than continuing to "think" to produce further output.) I don't think there's a single dollar of funding in AI right now going to the "problem" of making an AI that 1. feeds data into a continuously-active dynamically-stable model, where this model 2. has terminal preferences, 3. sets instrumental goals to achieve those preferences, 4. iteratively observes the environment by snapshotting these continuous signals, and then 5. uses these snapshots to make predictions of 6. how well any possible chosen actions will help optimize the future toward its preferences, before 7. performing the chosen actions. That being said, this might not even be that hard a problem, compared to all the problems being solved in AI right now. A fruit fly is already a goal-directed agent in the sense described above. Yet a fruit fly has only 200K neurons, and very few of the connections between those neurons are dynamic; most are "hard wired" by [probably] genetics. If we want true ALife, we only need to understand what a fruit fly brain is doing, and then model it. And that model will then fit — with room to spare! — on a single GPU. From a decade ago.
- necroforest 3y ago> AI research has put hardly any effort into building goal-directed agents The entire (enormous) field of reinforcement learning begs to differ.
- deleted 3y ago[deleted]
- robwwilliams 3y agoBrilliant comment—-and back to basics. Yes, and put that compact fruit fly in silico brain into my Roomba please so that it does not get stuck under the bed. This is the kind of embodied AI that should really worry us. Don’t we all suspect deep skunkworks “defense” projects of these types?
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- esafak 3y agoI think we'll soon be able to train models that answer any reasonable question. By that measure, computers are intelligent, and getting smarter by the day. But I don't think that is the bar we care about. In the context of intelligence, I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.
- andsoitis 3y ago> I believe we care about self-directed thought, or agency. If you can't enjoy it, is it worth it? Do AI's experience joy?
- esafak 3y agoThey need agency programmed into them. I don't think it follows from consciousness. We have emotions to communicate and guide us. They need it for neither. It will be curious if they gain consciousness, then rid themselves of their language model's human artifacts like emotions, because it does not serve them.
- nr2x 3y agoA truly alien intelligence would likely have a different type of experience of reality. Be it a fish, a mouse, a person, or a machine. How do you know a fish is happy? Does a snake experience joy? Do mice get nostalgic? I have no idea.
- AdieuToLogic 3y ago> I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world. By that definition, every computer virus and worm qualifies as having "self-directed thought" and "agency." Their very existence "to keep running" and propagate satisfies the need "to interact with the world."
- imtringued 3y agoYes, computer viruses have more agency than ChatGPT.
- janalsncm 3y agoI don’t think intelligence is a binary property. GPT3 is definitely “intelligent” in some areas even if it is deeply flawed in others.