0
$\begingroup$

I have atleast a terabyte of text data which I need to compress as efficiently as possible. It is file of english words and numbers (Not randomly arranged but forming meaningful syntax just as regular grammatical sentences). I got to know about CMIX compressor and its benchmarks (not too much understandable to me) and there was no short answer on google such as : cmix compressed a file of n GB to n GB OR n MB etc. So I want to know how much (how closely will it be able to) compression I will get if I use it on my 1 TB text data. And is there any other compressor better than cmix.

$\endgroup$
1
  • $\begingroup$ If you're talking about cmix by Byron Knoll, there are benchmarks on his web site here. Compressing enwik9 ($10^9$ bytes) to 1/9th its size takes about 1 week on unspecified hardware. At that rate, it would take about 20 years to compress 1 terabyte. Decompression is equally slow. My practical advice is to use Zstandard, Brotli, or LZMA/xz, which will compress by only 5:1, but in hours instead of years. $\endgroup$
    – benrg
    Commented Jan 19, 2022 at 0:10

1 Answer 1

1
$\begingroup$

There is no answer. It depends on the specific text file. If you want to know how much it will compress a specific file, the way to find out is to try compressing that file and see what you get. The compression ratio may vary from file to file, even for text data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.