4 ms·
It doesn't know that the vectors are tests. We don't encode a special "this is a test" bit in the vector when we test. There's no way for the matrix to figure o
by backpropaganda 9y ago
It doesn't know that the vectors are tests. We don't encode a special "this is a test" bit in the vector when we test. There's no way for the matrix to figure out that it's being multiplied to a test vector. This is just testing 101.
- yorwba 9y agoWhen I test software, I also don't tell it that it's running a test. It's not even intelligent, and yet it manages to misbehave in all kinds of ways that haven't been caught by the tests. Suppose you are working on a superintelligent AI, and your tests catch it destroying the contents of its sandbox. You fix that bug and move on to the next failing test, fix that too, and so on. When you run out of tests, you write more and fix those that fail, and so on. Eventually you can't come up with any failing tests anymore. When that happens, is the AI safe? I don't know. It passes all the tests, but all the previous versions had some bugs, so who is to say that this version is correct? Maybe your test scenarios just aren't realistic enough. Maybe they are realistic enough, but you haven't been looking hard enough for undesirable behavior. To run a superintelligent AI with any significant access to the outside world, I would like to see much stronger correctness guarantees than just a test suite, because tests can only show the presence of bugs, never prove their absence (paraphrasing Dijkstra).