I ran into this issue as well. As the person below mentioned, make this part of your teardown script. You can use ssh-keygen -R hostname to remove host from ~/.

I ran into this issue as well. As the person below mentioned, make this part of your teardown script. You can use ssh-keygen -R hostname to remove host from ~/.ssh/known_hosts.