3 ms·

You don't seem to understand how thise systems are trained if you believe it is by asking a lot of people and averaging answers. Plus there is a ton of work to

by ta988 3y ago

You don't seem to understand how thise systems are trained if you believe it is by asking a lot of people and averaging answers. Plus there is a ton of work to now combine LLMs with their excellent language abilities to more formal systems based on rules, ontologies and (fuzzy) logic. LLMs showed that this type of networks can likely solve one aspect if intelligence, the ability to handle language, that previous approaches were all extremely bad at. Doesn't mean that this approach is good for all other aspects of intelligence, but it turns out it is not too bad at information retrieval as well and a few other things. And other kind of networks also showed excellent abilities in other aspects like vision.

crabbone 3y ago
I work in HPC infra, so I know some stuff about how these network are trained, but more from a technical perspective, i.e. stuff like "how to set up MPI for a cluster of X nodes" or how to configure GPU drivers. I know basic stuff about how models are trained from another perspective too -- since I work with researchers, and my wife being an RSE who usually works with healthcare related research I'm also exposed to a lot of side effects of using neural network styled AI in practice. I don't claim to have a good knowledge of the ins and outs of the particular trend of the day though. I wouldn't be able to tell what the difference is between "deep learning" and "reinforcement learning", not with any authority or confidence. I did take two courses on AI though, long before the neural-network approach was a thing. I.e. my AI learning experience was focused on search algorithms, building knowledge bases etc. So, here, based on my experience of practical applications of LLMs, I can comment on things you wrote: > Plus there is a ton of work to now combine LLMs with their excellent language abilities to more formal systems based on rules Tons of publications doesn't mean this is a solved problem. As it stands now, there's a serious doubt this is a problem that can be solved. I have not seen any real system working in the world of medicine which would use LLMs for anything that can be encoded as rules. If anything, I've heard a lot about failures to implement anything remotely similar to this (which happened outside of academic medicine, or somehow slept through the cracks). I.e. chatbots suggesting suicide to people in distress, or suggesting to cut on carbs to young women struggling with anorexia and so on. > the ability to handle language, that previous approaches were all extremely bad at Well, now it's my turn to point fingers. You probably don't know much about previous approaches, and don't understand the problem well. It's true that since the term AI was coined substantial efforts went into understanding natural languages. But what came out of this work is a disillusionment, the understanding that natural languages suck at encoding knowledge and as such are a bad tool for building ontologies, or call them knowledge bases. So, it wasn't because older methods failed to parse language well. It was because we realized that there's no point in being good at it. It's a cute gimmick, if you can do it, and can be useful in harmless situations, like enabling rich interactions between NPCs and PCs in video games. But, if your goal is to understand how to make good decisions, it's more of an obstacle really. Like I said previously, but you seem to have ignored: yes, neural networks are useful, but not in the way a lot of people understand intelligence, or, at least, they fail to address the central and the critical questions about intelligence. Compounded by dishonest advertisement of their abilities and armies of low-quality practitioners this approach has a tendency to alienate people who are interested in more fundamental questions about the nature of intelligence.

by ta988 3y ago

crabbone 3y ago

I work in HPC infra, so I know some stuff about how these network are trained, but more from a technical perspective, i.e. stuff like "how to set up MPI for a cluster of X nodes" or how to configure GPU drivers. I know basic stuff about how models are trained from another perspective too -- since I work with researchers, and my wife being an RSE who usually works with healthcare related research I'm also exposed to a lot of side effects of using neural network styled AI in practice. I don't claim to have a good knowledge of the ins and outs of the particular trend of the day though. I wouldn't be able to tell what the difference is between "deep learning" and "reinforcement learning", not with any authority or confidence. I did take two courses on AI though, long before the neural-network approach was a thing. I.e. my AI learning experience was focused on search algorithms, building knowledge bases etc. So, here, based on my experience of practical applications of LLMs, I can comment on things you wrote: > Plus there is a ton of work to now combine LLMs with their excellent language abilities to more formal systems based on rules Tons of publications doesn't mean this is a solved problem. As it stands now, there's a serious doubt this is a problem that can be solved. I have not seen any real system working in the world of medicine which would use LLMs for anything that can be encoded as rules. If anything, I've heard a lot about failures to implement anything remotely similar to this (which happened outside of academic medicine, or somehow slept through the cracks). I.e. chatbots suggesting suicide to people in distress, or suggesting to cut on carbs to young women struggling with anorexia and so on. > the ability to handle language, that previous approaches were all extremely bad at Well, now it's my turn to point fingers. You probably don't know much about previous approaches, and don't understand the problem well. It's true that since the term AI was coined substantial efforts went into understanding natural languages. But what came out of this work is a disillusionment, the understanding that natural languages suck at encoding knowledge and as such are a bad tool for building ontologies, or call them knowledge bases. So, it wasn't because older methods failed to parse language well. It was because we realized that there's no point in being good at it. It's a cute gimmick, if you can do it, and can be useful in harmless situations, like enabling rich interactions between NPCs and PCs in video games. But, if your goal is to understand how to make good decisions, it's more of an obstacle really. Like I said previously, but you seem to have ignored: yes, neural networks are useful, but not in the way a lot of people understand intelligence, or, at least, they fail to address the central and the critical questions about intelligence. Compounded by dishonest advertisement of their abilities and armies of low-quality practitioners this approach has a tendency to alienate people who are interested in more fundamental questions about the nature of intelligence.