rsync of >4GB files

foolish one,

do you have any kind of troubles with rsync' ing huge [ >4GB ] files under modern-day linux? [ 32bit, 64bit, huge documents assistance activated ]? i've done some examinations on my very own in between 2 64bit boxes and also really did not have any kind of troubles moving 6 - 10GB files. to make examination detailed i transformed files, run rsync once more, examined md5 - all appears ok.

yet after i saw this bug record i obtained a little bit stressed. i did some looking yet have actually not located any kind of verification of the trouble.

many thanks for your ideas!

modify: documents system: ext3, reiserfs

3
2022-06-07 14:32:52
Source Share
Answers: 5

Nope, I spray 5 - 10GB VM photos making use of rsync regularly, never ever seen a trouble.

3
2022-06-07 14:57:00
Source

The biggest documents I have actually lately rsynced was 180GB which remained in a a set of directory sites having 15TB (I have actually created a set of manuscripts which can do the sync in parallel, I can relocate the information at concerning 3TB a hr )

3
2022-06-07 14:56:58
Source

No, we integrate 2 30TB information collections (constructed from files varying from 4 to 20GB) daily for months with rsync, no worry.

2
2022-06-07 14:56:48
Source

Depends on the filesystem that you are making use of. I've faced problem with FAT32 filesystems. I had a 200GB mobile harddrive (formatted as FAT32) and also was attempting to replicate a DVD.iso onto it. It really did not function create you can not have files more than 4. somethingGB in FAT32

4
2022-06-07 14:56:44
Source

The mistake record that you've connected to does not appear to be a 4GB+filesize relevant mistake. 429796854bytes is simply reluctant of 410Mb, and also it appears to be a transportation mistake as opposed to an rsync one. If.

I would certainly believe that the transportation link (probably SSH) has actually gone down, probably as a result of a lack of exercise timeout due to the fact that the CPU invested as long in between sends out due to the fact that it needed to do something like checksum a large documents, and also this is the factor that rsync reports a busted pipeline.

I'm certain I've made use of rsync efficiently on files over 4Gb in the past with 32bit customers and also web servers, and also at the very least as soon as where extra after that 4Gb was in fact moved as opposed to simply taken into consideration for transfer.

2
2022-06-07 14:53:49
Source