4 ms·

It’s so /weird/ (to me, at least) that ChatGPT uses what is essentially in-band signalling to parameterise/configure a session - the lack of actual hard-and-fas

It’s so /weird/ (to me, at least) that ChatGPT uses what is essentially in-band signalling to parameterise/configure a session - the lack of actual hard-and-fast settings that actually and directly control the system makes it feel like we’re trying to interface with something utterly otherworldly that no-one understands well enough to actually know what they’re doing - like when a 60-day coding-camp certificate holder can put together a neat React demo using a WASM port of OpenCV to do Canny edge-detection on a users’ webcam, but have no idea how to change the color of the output rendered edges. (Tenuous analogy, but it’s the best I could come up with after spending 10 minutes trying to come up with one…) Forgive my ignorance of modern AI techniques (I chose very different electives in my cs degree…), but why can’t they add “hard” model parameters that directly control the “thought-processes” going on? I assume it’s possible to have some kind of model representation of higher-level processes exposed as an interface… right?

colanderman 4y ago
> it’s possible to have some kind of model representation of higher-level processes exposed as an interface Basically it's not. The weights of neural networks -- which are trained, not designed -- are not ordered or separated in any organized or meaningful way. The only ways to approach this for a neural network would be either (a) manually tag the training set with some sort of "thought-process" annotations, so those can be trained as part of the model, or (b) train another neural network using the internal state of ChatGPT compared against such tags of output it generates. Either of these approaches encounters the scalability problem of creating those tags. One could imagine creating a model of thought to automate that process, but that I surmise is a _much_ more difficult problem than building ChatGPT (which has a relatively simple design). Another approach could be to train smaller neural networks against text that only represents certain types of thought, and then join those together into a larger network which is trained on general text (like ChatGPT). You then may have a more "ordered" final neural network product you could gain some insight into.

colanderman 4y ago

> it’s possible to have some kind of model representation of higher-level processes exposed as an interface Basically it's not. The weights of neural networks -- which are trained, not designed -- are not ordered or separated in any organized or meaningful way. The only ways to approach this for a neural network would be either (a) manually tag the training set with some sort of "thought-process" annotations, so those can be trained as part of the model, or (b) train another neural network using the internal state of ChatGPT compared against such tags of output it generates. Either of these approaches encounters the scalability problem of creating those tags. One could imagine creating a model of thought to automate that process, but that I surmise is a _much_ more difficult problem than building ChatGPT (which has a relatively simple design). Another approach could be to train smaller neural networks against text that only represents certain types of thought, and then join those together into a larger network which is trained on general text (like ChatGPT). You then may have a more "ordered" final neural network product you could gain some insight into.