3 ms·
What's interesting was how with GGC the model would spit out things relating to the enhanced feature vector, but would then in-context end up self-correcting an
by kromem 2y ago
What's interesting was how with GGC the model would spit out things relating to the enhanced feature vector, but would then in-context end up self-correcting and attempt to correct for the bias.
I'm extremely curious if as models scale in complexity if techniques like this will start to become less and less effective as net model representations collapse onto an enforced alignment (which may differ from the 'safety' trained alignment, but be an inherent pretrained alignment that can't be easily overcome without gutting model capabilities too).
I have a sneaking suspicion this will be the case.
- metadat 2y agoWhat's GGC in this context?
- dannyobrien 2y agoGolden Gate Claude
- rileyphone 2y agoIn that case there are two attractors - one towards the Golden Gate Bridge and one towards the harmless, helpful, honest assistant persona. Techniques as such probably get weirder results with model scale but no reason to think they get wiped out.
- coldtea 2y agoWhat if the Golden Gate Bridge is Main Kampf or something like that?
- wongarsu 2y agoThe preferred technique seems to still be to train a base model on any data you can get your hands on, and add the "safety" alignment as a second training step. As long as that alignment is a small fine tuning compared to the initial training I wouldn't be worried about the model losing the ability to be uncensored.