Let's say we're talking about a 1 GB video file. It's copy-pasted from hard disk D1 to the hard disk D2, then from D2 to D3, and so on, all using Windows. If we continue this process for like 1 million times, what would the resultant file look like? Do error bits accumulate and finally corrupt the file?
7
-
$\begingroup$ If you're mainly interested in what would happen when using Windows (or any other particular OS), this could be better for superuser. If, on the other hand, you're interested what would happen in general (and techniques to try and prevent bad errors), this can be on-topic, but then you should phrase the question as such (i.e. more general) $\endgroup$– Discrete lizard ♦Commented Feb 18, 2018 at 18:54
-
1$\begingroup$ The late Jim Gray wrote a paper on this topic. It's a fun and easy read. $\endgroup$– selbieCommented Feb 18, 2018 at 21:54
-
$\begingroup$ @selbie, that was really interesting. I want to know what happens when a file get too many error bits? Is it a similar scenario to a scratched/dirty DVD? $\endgroup$– asmaniCommented Feb 19, 2018 at 14:25
-
$\begingroup$ I've given you the general probability math below. But my main point is that once a single bit is toggled in error, the file is immediately considered corrupt. So I'm not sure what you mean by "too many error bits", because one bit in error means the file is already corrupted. $\endgroup$– selbieCommented Feb 22, 2018 at 15:17
-
$\begingroup$ @selbie I think the questioner means the degree of corruption is large enough the make the file unusable. Some files have low tolerance for corruption and will be immediately unusable (source code, binaries), but other files that are usually compressed lossy (images, video) can still be partially usable even under multiple erroneous bits. $\endgroup$– Discrete lizard ♦Commented Feb 22, 2018 at 15:52
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
4
Once a single-bit error is introduced into a file, the file is corrupt. File systems, disk drivers, and hardware on the disk itself have checksums, error correction codes, and facilities to detect bad sectors to limit the probability of write (or read) errors, but it's not 100% (but it better be close to 100%, otherwise my disk isn't reliable).
In general, the way to compute the probability of corruption:
Let's say the probability of a single bit error occurring during a file copy is P. (P should be really, really low, otherwise, the disk or media wouldn't be reliable).
And let's assume that a file has a size N measured in bits (e.g. a 1 GB file would be 8 billion bits or so).
So the probability of the first bit in the file not getting corrupted is: (1-P). And so it follows that the probability of all bits not getting corrupted during a transfer is (1-P)N
Now let's say you copy the file X times. Therefore the probability of the file not getting corrupted after X transfers is (1-P)NX. Or rather, the probability of corruption after X transfers is 1 - (1-P)NX
-
$\begingroup$ Aren't you assuming here that all operations result in an error, i.e. you ignore the fact that error correction sometimes works? You can say that that is a part of P, but I think it would be a lot better to explicitly model this. After all, the question is about to what extent the error correction will be effective. $\endgroup$– Discrete lizard ♦Commented Feb 22, 2018 at 15:49
-
$\begingroup$ Yes, I was implying that any error correction ability by the file system, storage system, or disk device was accounted for in P. $\endgroup$– selbieCommented Feb 22, 2018 at 23:53
-
$\begingroup$ I don't think this is a realistic model for error correction. Generally, an error correcting code can repair up to $R$ bits in a single file/block. Error correction triggers after each copy. This at least means that $N$ and $X$ shouldn't be interchangeable in the final formula, as we get that there is a probability $P(N,R)$ of an error in a single copy, which I doubt is equal to the one described and that the final probability of corruption is $1-P(N,R)^X$. $\endgroup$– Discrete lizard ♦Commented Feb 23, 2018 at 7:46
-
$\begingroup$ It's definitely not a model for error correction. It's a general model for error probability. I would encourage you to write your own answer. I might even upvote it myself... $\endgroup$– selbieCommented Feb 23, 2018 at 17:26