Strange data corruption with large files

I have 4 machines.
1 windows 2000 server with sata-ii drives
1 xp laptop sp2
1 xp laptop sp3
1 vista laptop sp1
All avg pro v8

When I copy a 10gb file from any machine to any other machine(copy/xcopy) and follow this with a "fc /b" all copies are identical.

However.... if I use ftp, rsync, or any other delta copying method I always get errors from "fc /b", usually 2 bits (02 -> 00) sometimes a few more.

When I overwrite the same file with copy /b or xcopy or even xxcopy, no problems, 100% ok.

Since this is happening to 4 different OS's and hardware systems I am excluding bad HD's and ram.

I have tried this with a smaller file like 2gb, no problems here.

I've been running tests for days now, recompiling rsync and other delta tools, added extra layers that run a md5 and sha1 checksum (who always report that the data written to disk matches both checksums) and still fc detects at least 2 wrong bits. xxcopy again and all is well.. slightly going insane here...

Any ideas?
5 answers Last reply
More about strange data corruption large files
  1. Nobody any ideas? could someone do a test?
    Ie. take a DVD and copy this as a ISO to PC-1
    take another DVD and copy this as a ISO to PC-2
    Install something like deltacopy and (r)sync one ISO with the other between pc's.
    Then run a fc /b between pc's ISO's.
  2. if the file still works then the extra two bits are probably just header information that the ftp program has put there.... don't worry about it.
  3. I wish... the bits are located at offsets like ecd09981 ie. anywhere beyond the 2.5g range. and no its not a 16/24bit limit problem.
  4. Just a quick twitter, I now suspect windows folder compression, its the only thing commonly used between pc's.
  5. Yep, confirmed, its windows disk compression doing this on all platforms, so not just a bug in home server, compress files larger then 1gb and you risk corruption... it took a few days to test this with 400gb, decompressing and repeat.
Ask a new question

Read More

Laptops Windows XP Apps