I don't think that is a problem with transfers over SSH. The compression wouldn't cause it to slow down speeds by a factor of 10 so there must be something else

I don't think that is a problem with transfers over SSH. The compression wouldn't cause it to slow down speeds by a factor of 10 so there must be something else that is cutting down the speed. I regularly (and only) transfer files over ssh/sftp and can manage to sature my 20mbit cable connection.