0
$\begingroup$

For a project I'm working on, I need to generate some data to test the Gzip compressor, and I wrote my own implementation of Markov chain for it, and it's a 2-level nested one.

But when I generate data using my script and feed it to Gzip, it always - without exception - expands the data, as if there's no redundancy in it.

What could be causing this? How can I modify my Markov chain so that it generates compressible data?

$\endgroup$

1 Answer 1

1
$\begingroup$

In the factory function that generates MC definition, each state has the probability of transitioning to other states that're uniformly random, I guess that this is the biggest problem.

Suggestion for improvement: Make the probability more sparse and skewed for each state for starters.

(I've made a new commit)

$\endgroup$
1
  • $\begingroup$ I confirm this did work. Although it occasionally generates un-compressible data, it's perfect for testing edge cases. $\endgroup$
    – DannyNiu
    Commented Dec 18, 2022 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.