If anything, I'd guess that when speaking English as second language, harder than knowing the accents on words would just be keeping track of all the exceptions

If anything, I'd guess that when speaking English as second language, harder than knowing the accents on words would just be keeping track of all the exceptions in pronunciation between words that you basically just have to memorize. Tough, though, taught, thought, through, thorough, throughout, etc.