Valid point and I clearly was under the misconception that Linux didn't care that much about the external interfaces. However, VCL is more akin the the kernel

Valid point and I clearly was under the misconception that Linux didn't care that much about the external interfaces. However, VCL is more akin the the kernel modules, relying on the internal APIs which are much less stable. When the semantics on how Varnish operates (with 4.0, the backend threads are completely new) and so the VCL language _needs_ to change in order to reflect this. The external API, made available through HTTP hasn't changed much.