Algorithm for estimating lossless compression factor

Question

Is there any algorithm for estimating how well certain array of bytes could be compressed in lossless manner?

Usage for me is to automatically choose "best" data reorder method without knowing what compression method end user is going to use. I don't need perfect/optimal result, I just need some kind of numerical result that I can use to compare different permutation runs.

For example string "AAAABBBBCCCCDDDD" should compress better than "ABDABBABCDCCDACD" but if I use e.g. Shannon Entropy for estimation then I get identical results.

Answer 1

In general the only way to know what the compression factor with a particular compression method is to compress the data using that compression method and see what the compression ratio is.

Some compression algorithms have different levels of compression (e.g., 0-9), which allow the algorithm to spend a lot more CPU time to try to get a bit better compression. You might be able to use the compression factor at a lower compression level as a predictor of the compression factor at a higher level, though this won't be perfect.