It's only the obvious solution in the sense that it's not really a solution but it comes close enough that memory often doesn't leak fast enough for it to matte

It's only the obvious solution in the sense that it's not really a solution but it comes close enough that memory often doesn't leak fast enough for it to matter for short running programs. A drop in garbage collector (like Boehm) makes your toy language instantly production ready. Basically code generation and garbage collection are the two things that make language implementation tedious. That's why so many languages are on top of Java or .Net. Implementing a language on those platforms is almost trivial because it solves both code generation and garbage collection.