Say I have 2 Gaussian sources X and Y. They are generated with mutivariate gaussian distribution with mean = [0, 0] and co-variance = [1, 0.9;0.9, 1] with 100 observations
That is: means is zero and both X and Y are correlated with 0.9 and in total there are 100 variables for each of them.
I wanted to convert these correlated Gaussian variables X and Y into Binary, such that total bits for X and Y should be near to differential entropies H(X) and H(Y)
. This could be achieved using some best source coding schemes. I tried to find open source LZW compression code. But couldn't find one for integers (mostly are based for text compression)
But the point is after converting them into binary and arranging them as long codeword, I want their Hamming distance as small as possible. How to do that?
Because they are correlated, their hamming distance should be less.I am not able to achieve this. any help would be appreciated..
Edit part:
[1] For instance, 255≈256 but 011111111 and 100000000 are as different. This can be solved by using gray coding, then it will be having hamming distance of 1.
[2] I have tried using lloyd max quantizer(say 4 bit) on gaussians and then converting their indices into bits(gray coding). But still hamming distance is very high.
[3] 100 observations are just an example.(not significant)
[4] H(x)
is the differential entropy for X and it is mathematically equal to(1/2)(ln((2*pi*e)(det(cov_matrice))))
.
Thank you for all responses
1000000
and0111111
are very similar numbers but have maximal Hamming distance. $\endgroup$ – Raphael♦ Sep 12 '17 at 16:44