The primary driving force was nativism. Even right up to the end the UK wanted to remain in the common market, but part of being in the common market is freedom

The primary driving force was nativism. Even right up to the end the UK wanted to remain in the common market, but part of being in the common market is freedom to migrate. It also exposed the loss of influence the UK had in the EU. Many provisions were carved out for them in the 80s. But when it came to the EC the EU was not making any exceptions for them this time.