Bad analogy, but let's go along with it. The open source license already says you can go into the house and do whatever you want, including breaking something

Bad analogy, but let's go along with it. The open source license already says you can go into the house and do whatever you want, including breaking something and kicking you out. Your additional instruction to not break something goes against the rights initially granted. To make it worse, one is covered by copyright law and the other by business contract law.