2 ms·
> The basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice. By going in that direction, I feel like
by n4r9 3y ago
> The basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice.
By going in that direction, I feel like they put the nail in the coffin. Why would I use their website when I can just ask ChatGPT directly?
I think they had one other option, which was to identify the fundamental limitations of LLMs, and find a way to demonstrate value over and above.
- cm2187 3y agoPlus where will chatgpt take new stuff from?
- famahar 3y agoWhatever is new will probably be designed for chatgpt to understand first. Maybe people won't bother using your tool if it can't be used without a AI chat interface helping. The same way I won't learn a programming language if the documentation is bad. Tool makers knowing this will make chatgpt specific technical documents that can be easily parsed by an LLM.
- cm2187 3y agoStackoverflow is about expert programmer answering questions from novice programmers (it can be novice on a specific tech, not necessarily junior devs). How will chatgpt learn from expert programmers, who don't need chatgpt to code something up.
- famahar 3y agoMy theory is that experts will be hired to train specific models. Documentation will be written to be understood my an LLM chat interface. Edge cases and tutorials will be pre-made and then fed to the model. Technical writing might be skill that will be in huge demand.
- ilaksh 3y agoPeople were already just asking ChatGPT. I don't believe there is any such fundamental limitation, especially when you look say two years down the road and don't limit yourself to language-only models or the AI models of today. Already GPT-4 has audio and image input. Good human programmers today have better abilities and more current knowledge overall than GPT-4. But it's a matter of degree, and a matter of time. What types of limitations do you suppose? And again, we have to anticipate that these technologies evolve rapidly. LLM is a misleading term in my opinion for a fundamentally multimodal model. And totally new types of models or model integrations will be likely be coming out over the next few years that will make current systems obsolete. Such as combining AlphaZero-type systems with language models.
- n4r9 3y ago> What types of limitations do you suppose? I don't see the capacity for LLMs to perform novel and creative reasoning tasks. They're limited by virtue of being a pattern-matching system without the ability to make genuine causal or logical inferences. I wrote here about my recent experiments with ChatGPT - https://news.ycombinator.com/item?id=37583333 https://news.ycombinator.com/item?id=37583333 I'm vaugely interested to know how GPT4 deals with questions like these, but I'm not willing to pay to find out.
- ilaksh 3y agoLol. So even the dumb version actually did your task, just not in a very convincing or "genuine" way. Pattern matching can be used for inference and GPT-4 does it very effectively. 3.5 turbo is relatively brittle and maybe 20 IQ points lower. But still, it did your task. Anyway, it will only be a year or two before things like tree search over problems spaces are explicitly integrated into models or systems similar to this. Deep Mind may have already done something like that.