Debugging is by far the most difficult thing when working with AI algorithms because the output is rarely deterministic and that makes it hard to tell the diffe

Debugging is by far the most difficult thing when working with AI algorithms because the output is rarely deterministic and that makes it hard to tell the difference between actual bugs and a bad selection of hyperparameters. You're also typically working huge arrays of numbers so unless you have some way to visualise what's happening it's also difficult to pin point where the algorithm is doing something it shouldn't be. My main criticism of learning about algorithms using a framework is that they handhold so much with optimisation techniques and accessible APIs that you can know very little about what's actually happening and build something that works. When writing AI algorithms just knowing how to implement in theory is rarely ever enough. The difficulty is always in the debugging and optimisation techniques.