The real problem is that it attempts to break down the whole image in a single transform, rather than splitting it into blocks. There is nothing wrong with an

The real problem is that it attempts to break down the whole image in a single transform, rather than splitting it into blocks. There is nothing wrong with an optimized set of basis functions, and tests have actually shown that an overcomplete set of basis functions does actually give better results than a complete transform like the DCT. Albeit at higher computational cost--an RD-optimal overcomplete transformation, even if basis functions are already known, is NP, related to subset-sum IIRC. Even if you're not going overcomplete, tests have also shown that optimized basis functions are useful. However, there are two catches here. The first is that if you try to make an optimal basis function set over a huge amount of real-world input data, you will get the DCT! The second is that the problem of creating an optimal classifier--that is, to split inputs into groups which would get their own set of basis functions (and then optimize for each of those groups)... is also NP. Though it might be possible to approximate it efficiently with K-means. However, regardless, you want to perform transforms on relatively uniform areas of the image. Thus, you want the individual transforms to be small enough to cover relatively uniform areas and localize nonuniformity as much as possible. That's why JPEG uses 8x8 blocks, not a full-image DCT. That's also why H.264 has two transform sizes (and effectively a third if you count the heirarchical transform in i16x16 blocks): so it can adapt to uniform areas of different size.