I believe that for non-naive English speakers like me these "I'm lying" phrases are very annoying. Mainly because we still translate sentences into our own moth

I believe that for non-naive English speakers like me these "I'm lying" phrases are very annoying. Mainly because we still translate sentences into our own mother language (or some kind of proto-language) and then try understand the meaning. For example, "Look, to be honest.." - what does it mean? You were not honest before? Or "I'm not saying..." - but you are saying? Why do you need that introduction? What do you want to communicate with that? I believe that native English speakers swallow and ignore these phrases more easily. Am I wrong here? Do other non-native English speakers feel the same?