3 ms·

Earlier I was thinking about camera components for an Arduino project. I asked ChatGPT to give me a table with columns for name, cost, resolution, link - and to

Earlier I was thinking about camera components for an Arduino project. I asked ChatGPT to give me a table with columns for name, cost, resolution, link - and to fill it in with some good choices for my project. It did! To describe this as "autocomplete word salad" seems pretty insufficient. Autocomplete can use a search engine? Write and run code? Create data visualizations? Hold a conversation? Analyze a document? Of course not. Next token prediction is part, but not all, of how models are engineered. There's also RLHF and tool use and who knows what other components to train the models.

snowwrestler 2y ago
> Autocomplete can use a search engine? Write and run code? Create data visualizations? Hold a conversation? Analyze a document? Of course not. Obviously it can, since it is actually doing those things. I guess you think the word “autocomplete” is too small for how sophisticated the outputs are? Use whatever term you want, but an LLM is literally completing the input you give it, based on the statistical rules it generated during the training phase. RLHF is just a technique for changing those statistical rules. It can only use tools it is specifically engineered to use. I’m not denying it is a technology that can do all sorts of useful things. I’m saying it is a technology that works only a certain way, the way we built it.