3 ms·

Your understanding is very outdated. Go take a look at some of the things people are doing with LangChain to get a sense of what's possible today and what will

by cs702 4y ago

Your understanding is very outdated. Go take a look at some of the things people are doing with LangChain to get a sense of what's possible today and what will likely be possible in the very near future. LLMs are normally used in a zero-shot setting, without any kind of fine-tuning.

ezst 4y ago
> Your understanding is very outdated. And yours may be very optimistic. LLMs are not knowledge bases, and ChatGPT has proven it times and times again by dreaming stories when asked for facts. Even though you could engineer a way to structure answers that seem like lines of reasoning, there is no way to 1- prove that observations aren't entirely made up (the LLM doesn't "know", and worse, this model is not designed to evaluate how much it deviates from a baseline, i.e. what it invented vs. what it thinks it knows for certain) and 2- there is no formal evidence that this way of structuring answers is sound logic (again, you can fool ChatGPT into logic errors that a high schooler would discern, and that makes sense solely based on the fact that LLMs don't work like formal systems by composing axioms and proofs ; that at the moment is impossible because of 1).
mark_l_watson 4y ago
I agree with your optimism, although I can understand if people want to argue with this optimism. I think that we are seeing exponential improvements in deep learning, specifically in LLMs. A few months ago, I would see about one new demo or great paper a day that really impresses me. Now I see many great new demos, or products, or papers a day. In the 1980s we would build commercially useful models using a few layers with backdrop with small layer sizes. We built our own Harvard Architecture hardware for forward feeds and backdrop, a whopping 5 million flops/second. That was fun, but I prefer modern tools, thank everyone very much! BTW, I am working on a short book that uses LangChain, Llama-Index (used to be called GPT-Index), and a few other libraries to solve a few problems that I find interesting.

by cs702 4y ago

ezst 4y ago

> Your understanding is very outdated. And yours may be very optimistic. LLMs are not knowledge bases, and ChatGPT has proven it times and times again by dreaming stories when asked for facts. Even though you could engineer a way to structure answers that seem like lines of reasoning, there is no way to 1- prove that observations aren't entirely made up (the LLM doesn't "know", and worse, this model is not designed to evaluate how much it deviates from a baseline, i.e. what it invented vs. what it thinks it knows for certain) and 2- there is no formal evidence that this way of structuring answers is sound logic (again, you can fool ChatGPT into logic errors that a high schooler would discern, and that makes sense solely based on the fact that LLMs don't work like formal systems by composing axioms and proofs ; that at the moment is impossible because of 1).

mark_l_watson 4y ago

I agree with your optimism, although I can understand if people want to argue with this optimism. I think that we are seeing exponential improvements in deep learning, specifically in LLMs. A few months ago, I would see about one new demo or great paper a day that really impresses me. Now I see many great new demos, or products, or papers a day. In the 1980s we would build commercially useful models using a few layers with backdrop with small layer sizes. We built our own Harvard Architecture hardware for forward feeds and backdrop, a whopping 5 million flops/second. That was fun, but I prefer modern tools, thank everyone very much! BTW, I am working on a short book that uses LangChain, Llama-Index (used to be called GPT-Index), and a few other libraries to solve a few problems that I find interesting.