"glue them together as required in my application code" This can easily be the source of problems. Bad compositions of crypto primitives are a common source o

"glue them together as required in my application code" This can easily be the source of problems. Bad compositions of crypto primitives are a common source of vulnerabilities in cryptosystems. A textbook example is MAC-then-ENCRYPT, which is not secure in general. Another common example is over-use of a cryptographic key, which can easily happen when underlying primitives are composed badly (asking for too many bytes from a PRG, or encrypting too much data in GCM mode, etc.). "never really screw with asymmetric crypto primitives directly" You might not be matching your crypto to your application requirements if you can write off a whole swath of primitives like that, or else you only deal with a very specific setting in which symmetric cryptography is sufficient.