Though in a way, this feels similar to things like garbage collection, disk defragmentation, or even query planning. Yes, you could build libraries that do thes

Though in a way, this feels similar to things like garbage collection, disk defragmentation, or even query planning. Yes, you could build libraries that do these sorts of things for you, but in all likelihood the LLM providers will embed custom-built versions of them that have been battle tested and trained thoroughly to interop well with the corresponding LLM. So whole there could still be an ecosystem, it would likely be a fairly niche thing for very specific use cases or home-grown LLMs. Maybe something like the equivalent of AWS Firecracker for whatever the equivalent of AWS Lambda is in the future LLM world.