0
$\begingroup$

When compressing multiple files, is it more efficient to (1) first compress each individual file separately and then concatenate/merge the results, or (2) first concatenate/merge the files into a single archive and then compress that archive? Which one will be faster, in terms of running time?

From what I understand back in the days (.tar.gz, .tar.Z) when a number of files were archived the files were usually "merged together", and then the resulting file was compressed. But now on newer systems with newer software and algorithms like WinRAR or win ZIP, the files are individually compressed and then the resulting compressed files were merged together to form an archive. Is one of those approaches faster than the other?

It seems like (2) might be faster. Let's take for example huffman compression. If we compress different files separately than we need to remember a dictionary or some sort of table where we know how many times or with what percentage a certain word appears in the file. This table takes additional space for every file stored. If we compress a huge file than we store less tables. Is this right?

Improve this question
$\endgroup$
6
  • $\begingroup$ I've edited your question to distinguish it from cs.stackexchange.com/q/60588/755. But now it has a different problem: there probably is no single answer that applies to all compression routines. So which compression algorithm specifically are you talking about? Is there some additional context or motivation that you haven't mentioned? $\endgroup$
    – D.W. ♦
    Nov 9, 2016 at 1:15
  • $\begingroup$ No, there isn't an additional context. I just want to know if there are certain compression algorithms that perform faster than other when used in one of the two modes mention in my question and why. $\endgroup$
    – yoyo_fun
    Nov 9, 2016 at 1:21
  • $\begingroup$ (The description of the additional information for decompressing "static Huffman" is a, well, transcription from a comment by yoyo_fun.) $\endgroup$
    – greybeard
    Nov 9, 2016 at 5:40
  • $\begingroup$ (The replaced formulation efficient (in terms of space and processing time) might have meant something else: in the context of complexity analysis, space looks RAM required for processing, with data compression, it may have referred to effectivity: output size: please comment.) $\endgroup$
    – greybeard
    Nov 9, 2016 at 5:42
  • $\begingroup$ By and large, general purpose lossless compressors exploit self-similarity. Differences in processing files individually vs. combined come from starting processing a new file with previous vs. zero knowledge: If the file is/starts similar to something remembered, compression should be more efficient as well as more effective. If the current file is dissimilar, prior knowledge may be detrimental and need to be "unlearned". $\endgroup$
    – greybeard
    Nov 9, 2016 at 5:50

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.