4 ms·
No, just a programmer that has to apply math. Transformers can only do what is called weak negation. if (not (goal X)), then (assert not x) One example I hav
by nyrikki 3y ago
No, just a programmer that has to apply math.
Transformers can only do what is called weak negation.
if (not (goal X)), then (assert not x)
One example I have seen is:
"you can cross if you have no information on a train coming”
VS:
"you can cross if you have evidence that no train is coming"
See the difference?
As to what 'transformers' can encode is ambiguous and depends on many factors but here is some light reading.
https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00493/112604/Saturated-Transformers-are-Constant-Depth https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00493...
https://arxiv.org/abs/2204.06618 https://arxiv.org/abs/2204.06618