The hard part is the editor. You can make a nice 3D renderer, learn shaders, bring in a physics library, make a bunch of models and textures, write a lighting

The hard part is the editor. You can make a nice 3D renderer, learn shaders, bring in a physics library, make a bunch of models and textures, write a lighting system, add a system of animation, make an actor/AI system, create 2D UI helpers for menus/HUDs, work out a file format for storing and loading state... ...And then realize that it's very difficult to create a detailed world when you can't see it and manipulate it in realtime. D'oh.