4 ms·
To be fair, it is good to know that it disobeys simple instructions like "don't examine my git history" far more than other models. (It should of course be a di
by numeri 4mo ago
To be fair, it is good to know that it disobeys simple instructions like "don't examine my git history" far more than other models. (It should of course be a different benchmark, so as not to conflate things.)
It's not a great sign for alignment.
- bensyverson 4mo agoAgreed, alignment is just a separate issue that a vuln fixing benchmark doesn't need to be testing.