It does but its hard to quantify, if your site loads in 4 seconds but after a change the sites loading time becomes 8 seconds you will lose some people who will

It does but its hard to quantify, if your site loads in 4 seconds but after a change the sites loading time becomes 8 seconds you will lose some people who will be annoyed and wont sign up. Google bots might prefer another site which loads faster compared to your site which will affect SEO.