I think the post missed a great opportunity to point out that rate distortion theory is the fundamental problem that we are stuck at -- be it viewed as a inform

I think the post missed a great opportunity to point out that rate distortion theory is the fundamental problem that we are stuck at -- be it viewed as a information theory problem, machine learning problem, statistics problem, signal processing problem ... This. is. the. holy. grail. Note for the lossless case with discrete alphabets we have polynomial time universal algorithms for source coding. No matter what the unknown distribution, these algorithm (eventually) guarantee the optimal possible performance had we known the distribution ahead of time. No need to assume a parametric family of distributions etc etc. We have not been able to solve this for continuous alphabet/ lossy / rate distortion case. We don't know of any universal algorithm that's efficient. If we knew how to do build this we would solve ML/signal processing/statistics etc., and gone home (to build other things). Even an impossibility result would help -- that says you cannot have an efficient algorithm unless, say P=NP.