Just start what you like to do. I don't know a lot about math (I still don't) but I always wanted to write a 3D render engine. I also was afraid of the complex

Just start what you like to do. I don't know a lot about math (I still don't) but I always wanted to write a 3D render engine. I also was afraid of the complexity. But then I figured I didn't know anything about it's complexity because I never tried. So I started with Processing and wrote an photo-realistic ligh-tracer. Very slow but very very fun. Then I wrote a multi-core version which was a little faster. Then an path-tracer which was faster. Then an exporter for Blender. Then I ported the project to Java. Is it as good as current render engines? No! But I don't care because I learned a lot and had a lot of fun. Then I liked to have a CNC router. But I could not afford one. So I just built one and it worked! Moral of the story: just do it. You will fail sometimes but who cares? You will always learn a lot.