What causes data corruption?

October 1, 2004 12:12 AM

Causes of data corruption in computers/networks. What modes of data transfer are most likely to cause corruption in a file, and what safeguards exist to prevent that from happening? See more now. _{See it inside.}
posted by shoos to computers & internet (8 comments total)

For example, which of these is most likely to cause a data error:
- Copying a file from one computer to another across a small home network.
- Downloading from a physically close server on the net (or from one on the other side of the world).
- To a CDR.
- To another location on the same drive.

On top of this, does the available computing power of the machines in question make a significant difference?
posted by shoos at 12:18 AM on October 1, 2004

Networking protocols are usually smart enough to use TCP or another correctable stack. Errors are detected and packets re-requested on the fly.

CDs can be bad, so these can be corrupted. Had this happen MANY times, especially with both a bad burner and crappy discs.

HDs can be bad, so the above are possible. Had this happen too.

HD cables and controllers can be bad, and can cause the above also. Had that happen too!!!

#1 best way to ensure bad data: Ymodem-G over 9600 baud modem.

On top of this, does the available computing power of the machines in question make a significant difference?

No.
posted by shepd at 12:23 AM on October 1, 2004

they should all either work or fail (ie refuse to copy).

so if you get corruption, it's either a software error (bug) or hardware failure (crash), not "noise".

i would guess that the most likely source of errors is hard disk failures. then perhaps poorly written device drivers. so copying involving disks, or devices that are not in widespread use, is most likely to give an error.

(for very large data rates there is a bug in how tcp over ip is implemented (fragment id key too short), but you're not going to hit it (it's just starting to become a problem moving terrabytes over fast links)).

incidentally, i'd love to be corrected on the above because i get the impression most people believe errors in transmission do occur with protocols like tcp (which should detect and correct or fail to transfer), but i can't see a good technical reason why.
posted by andrew cooke at 6:45 AM on October 1, 2004

so if you get corruption, it's either a software error (bug) or hardware failure (crash), not "noise".

That's actually not 100% true. In rare cases, poorly shielded networking cable can experience data corruption if it's run parallel and close to other RF-emitting wires (speaker wires, phone lines, electric lines, etc).

I forget exactly why this is, though, but it has to do with the fact that any cable like that is essentially just running pulses of electrons down it, and if you've got two running parallel they can influence each other if not properly shielded.

Other than that, what shepd said. See also 'poorly written network applications utilizing UDP'.
posted by cyrusdogstar at 7:54 AM on October 1, 2004

Two caveats: While it's possible to see data corruption due to 'noise', it's still incredibly rare, ao andrew cooke is still mostly right in that it's not really something to worry about.

And, my final statement was not implying that all network apps using UDP are poorly written, simply that if one uses UDP, it doesn't do error correction, so if the application isn't written correctly it may be prone to transmission errors.
posted by cyrusdogstar at 8:42 AM on October 1, 2004

my point wasn't that noise doesn't occur, but that the protocols will detect it and re-send the data or fail. you should not get a program finishing without an error and bad data. more exactly, the maths involved in the detection algorithms makes any possibility incredibly unlikely.
posted by andrew cooke at 12:43 PM on October 1, 2004

In response to shepd's last point: does that mean that while I'm in the process of transferring a file to my computer, if I'm simultaneously playing Half Life or Photoshopping a 300 MB file, it won't increase the chances of corruption of that file?

andrew cooke: so if you get corruption, it's either a software error (bug) or hardware failure (crash), not "noise".
Do you mean that if I do get corruption after transferring a file, the file will be non-functional, and won't just contain, say, audible distortion in the case of an MP3 file, or a missing letter in the case of a text file? (no, I don't know too much about how computers function!)
posted by shoos at 2:53 PM on October 1, 2004

i mean that you should either get a perfect file, or no file at all.

if that is not the case then all bets are off (the consequences of software bugs and hardware failures are not predictable).
posted by andrew cooke at 3:50 PM on October 1, 2004