Perhaps it can work through multiple stages: ICL -> prompt/context optimization (*) -> prefix tuning / KV distillation -> context distillation. *: it is possi

Perhaps it can work through multiple stages: ICL -> prompt/context optimization (*) -> prefix tuning / KV distillation -> context distillation. *: it is possible to measure how much part of a prompt helps with a task e.g. measuring change in entropy