Yes, the conventional wisdom is that C is less secure than high-level languages. If you're talking memory management, that's correct. DJB does nothing but stack

Yes, the conventional wisdom is that C is less secure than high-level languages. If you're talking memory management, that's correct. DJB does nothing but stack variables for NaCl to avoid performance and potential for memory leaks. But for side channels, C is almost too high level. For example, it takes assembly code (cpuid) to figure out cache configuration. Even worse, a language like Java or python gives you no control over the memory layout of the key in RAM. So the right combo in my opinion is a carefully tuned C implementation (possibly with a little assembly or compile-time tweaking). Then add your language bindings to call it from a HLL.