1
$\begingroup$

Say I have 2 Gaussian sources X and Y. They are generated with mutivariate gaussian distribution with mean = [0, 0] and co-variance = [1, 0.9;0.9, 1] with 100 observations That is: means is zero and both X and Y are correlated with 0.9 and in total there are 100 variables for each of them.

I wanted to convert these correlated Gaussian variables X and Y into Binary, such that total bits for X and Y should be near to differential entropies H(X) and H(Y). This could be achieved using some best source coding schemes. I tried to find open source LZW compression code. But couldn't find one for integers (mostly are based for text compression)

But the point is after converting them into binary and arranging them as long codeword, I want their Hamming distance as small as possible. How to do that?

Because they are correlated, their hamming distance should be less.I am not able to achieve this. any help would be appreciated..

Edit part:

[1] For instance, 255≈256 but 011111111 and 100000000 are as different. This can be solved by using gray coding, then it will be having hamming distance of 1.

[2] I have tried using lloyd max quantizer(say 4 bit) on gaussians and then converting their indices into bits(gray coding). But still hamming distance is very high.

[3] 100 observations are just an example.(not significant)

[4] H(x) is the differential entropy for X and it is mathematically equal to(1/2)(ln((2*pi*e)(det(cov_matrice)))).

Thank you for all responses

Improve this question
$\endgroup$
9
  • $\begingroup$ "Because they are correlated, their hamming distance should be less." -- how so? 1000000 and 0111111 are very similar numbers but have maximal Hamming distance. $\endgroup$ – Raphael♦ Sep 12 '17 at 16:44
  • $\begingroup$ Have you tried using a standard compression algorithm? What's the result, and why does it not satisfy you? $\endgroup$ – Raphael♦ Sep 12 '17 at 16:46
  • $\begingroup$ My gut feeling here is that even though the random variables are strongly correlated, the binary representations are mostly independently random (it would be interesting to investigate this rigorously), and so there is not much to gain by compression. $\endgroup$ – Raphael♦ Sep 12 '17 at 16:46
  • $\begingroup$ I'm not sure why the fact that you have 100 observations is relevant. This is really about just one observation. $\endgroup$ – Yuval Filmus Sep 12 '17 at 16:49
  • $\begingroup$ What do you mean by $H(X)$? How many bits do you want to extract from each sample? $\endgroup$ – Yuval Filmus Sep 12 '17 at 16:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.