What if the hit nose is not a transaction? What if an individual swings their arms without care, and your nose merely happens to be in the way? I might get mad

What if the hit nose is not a transaction? What if an individual swings their arms without care, and your nose merely happens to be in the way? I might get mad at you when I find an obstruction to my swinging. You might equally get mad at me for having hit you. Deciding between these constitutes deciding who has a right in that situation. In the example, you have a right to not be hit. Therefore, I have no right to swing my arms about if it conflicts with that right. However this decision is completely arbitrary; a society could just as easily exist that decides that I have a right to swing my arms about, and you have a responsibility to get out of my way.