There is a fair amount of existing work related to planning in continuous and unbounded spaces under uncertainty. It seems likely some of the existing technique

There is a fair amount of existing work related to planning in continuous and unbounded spaces under uncertainty. It seems likely some of the existing techniques combined with modern language models could be quite effective. A couple entry points: Deep Reinforcement Learning with an Unbounded Action Space - https://arxiv.org/abs/1511.04636v3 https://arxiv.org/abs/1511.04636v3 Efficient Planning in a Compact Latent Action Space - https://arxiv.org/abs/2208.10291 https://arxiv.org/abs/2208.10291