Having played with OpenGL, DirectX and XNA (which is really just a managed wrapper around DirectX as are other managed libraries such as SlimDX), I'd suggest th

Having played with OpenGL, DirectX and XNA (which is really just a managed wrapper around DirectX as are other managed libraries such as SlimDX), I'd suggest that learning any one of the above is really as low level as you want to go - At a basic level they simply provide an interface to the graphics hardware by allowing you to create and manage the vertex/index buffers and shaders which you load up on your graphics card which then does all the work of transforming the lists of vertices into what you see on the screen. With any of these low level libraries you will still need to develop the 'engine' itself which will deal with scene management, lighting, ai, physics, animation etc. You can still have a load of 'fun' and learn a huge amount by writing things like your own deferred lighting engine (which is what i initially did, there are plenty of tutorials online and it's a good way of learning some of the vector/matrix algebra which you'll need and pretty satisfying when you get first get hundreds of lights on your models), Even if you use libraries for lighting/physics, you'll still need to learn HLSL/GLSL for shader programming which is where a lot of the cool stuff is being done these days especially with the advent of geometry shaders.