3 ms·
> Tutoring me on how to use poorly documented APIs Good luck with that. GPT-4 is _incredibly_ bad at Metal (most under-documented GPU api I've ever worked wit
by nice_byte 3y ago
> Tutoring me on how to use poorly documented APIs
Good luck with that.
GPT-4 is _incredibly_ bad at Metal (most under-documented GPU api I've ever worked with), but more importantly it's _even worse_ at Vulkan, which is complete opposite - it's meticulously documented.
The stuff LLMs are good at is the stuff that a lot of people use and talk about all the time, in public.
There's just not a lot of public discussion (with examples) about e.g. how to properly use Vulkan in a production application, and the LLM itself can't "reason" within the framework of the detailed specification, so most things it outputs is wrong.
In the long run though, a very likely outcome is that tools that the LLMs have a poor "understanding" of just wither and die, or are relegated to being something only 3.5 people in the world use. I can see a future where literally all software is written in python just because that's the language that OpenAI's LLMs happens to know best.
- ilaksh 3y agoIt has an 8kb (or 32kb if you are lucky) context. Try giving it the relevant documentation examples before asking a question.
- nice_byte 3y agothe point the grandparent was making was that GPT-4 can teach you how to use an under-documented API. that means the relevant docs are either sparse or non-existent, otherwise there wouldn't be a need to ask GPT-4 in the first place.