Rsync usual commands

Details: Written by: Alex; Category: Linux; Published: 22 May 2023; Hits: 349

Article Index

Syntax of rsync command:

Local Sync: # rsync {options} {Source} {Destination}

Remote Sync pull: # rsync {options} <User_Name>@<Remote-Host>:<Source-File-Dir> <Destination>

Remote Sync Push: # rsync <Options> <Source-Files-Dir> <User_Name>@<Remote-Host>:<Destination>

Some of the commonly used options in rsync command are listed below:

-v, –verbose	Verbose output
-q, –quiet	suppress message output
-a, –archive	archive files and directory while synchronizing ( -a equal to following options -rlptgoD)
-r, –recursive	sync files and directories recursively
-b, –backup	take the backup during synchronization
-u, –update	don’t copy the files from source to destination if destination files are newer
-l, –links	copy symlinks as symlinks during the sync
-n, –dry-run	perform a trial run without synchronization
-e, –rsh=COMMAND	mention the remote shell to use in rsync
-z, –compress	compress file data during the transfer
-h, –human-readable	display the output numbers in a human-readable format
–progress	show the sync progress during transfer

Example 1: Copy or Sync files locally (rsync -zvh)

# rsync -zvh /home/docs/Azure Cloud Management.pdf /opt/backup
Azure Cloud Management.pdf
sent 4.09M bytes  received 35 bytes  2.73M bytes/sec
total size is 6.15M  speedup is 1.50
#

In above we have used the options like -z for compression, -v for verbose output and -h for human readable output.

Example 2: Copy or Sync directory locally (rsync -zavh)

Let’s assume we want to copy or sync docs user’s home directory to /opt/backup folder, execute the below rsync command with options -zavh:

# rsync -zavh /home/docs /opt/backup
sending incremental file list
docs/
docs/.bash_logout
docs/.bash_profile
docs/.bashrc
docs/Azure Cloud Management.pdf
sent 4.09M bytes  received 96 bytes  8.19M bytes/sec
total size is 6.15M  speedup is 1.50
#

Example 3: Copy files & directories recursively locally (rsync -zrvh or rsync -zavh)

Let’s assume we have multiple files and directories inside docs user home directory, use below rsync command to copy files and directories recursively, either use -a or -r option to copy files and directories recursively.

Note : In rsync command -a option is used for archiving during the copy or sync and apart from archiving -a option is also used for followings:

recursively copy files and directory
copy symlinks as symlinks
preserve permissions
preserve group
preserve modification time
preserve ownership

Then:

# rsync -zrvh /home/docs /opt/backup
or
# rsync -zavh /home/docs /opt/backup

Example 4: Copy or sync files and directories from local to remote system

Let’s suppose we want to copy the folder “/home/docs/tech” from local machine to remote machine (192.168.1.29) under the /opt

# rsync -zarvh /home/docs/tech This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
…………………………………………………………
tech/OTRS/
tech/OTRS/Database-Selection-OTRS-Installation-CentOS7.jpg
tech/OTRS/Install-OTRS-Accept-License.png
tech/OTRS/Install-OTRS-CentOS7-RHEL7.jpg
tech/OTRS/mysql-secure-installation-centos7-part2.jpg
sent 34.85M bytes  received 2.65K bytes  6.34M bytes/sec
total size is 37.77M  speedup is 1.08
……………………………………………………………
#

Example 5: Copy or Sync files and directories from remote machine to local system

Let’s suppose we want to copy files and directories from remote machine(192.168.1.29) to our local system, in the below example I am copying remote folder “/opt/rpms_db” in my local machine under /tmp folder

# rsync -zarvh This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt/rpms_db /tmp
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
receiving incremental file list
……………………………………………………………
rpms_db/
rpms_db/httpd-2.4.6-88.el7.centos.x86_64.rpm
rpms_db/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm
rpms_db/postfix-2.10.1-7.el7.x86_64.rpm
rpms_db/pytalloc-2.1.13-1.el7.x86_64.rpm
rpms_db/samba-4.8.3-4.el7.x86_64.rpm
rpms_db/samba-client-libs-4.8.3-4.el7.x86_64.rpm
rpms_db/samba-common-4.8.3-4.el7.noarch.rpm
rpms_db/samba-common-libs-4.8.3-4.el7.x86_64.rpm
rpms_db/samba-common-tools-4.8.3-4.el7.x86_64.rpm
rpms_db/samba-libs-4.8.3-4.el7.x86_64.rpm
sent 484 bytes  received 15.45M bytes  1.07M bytes/sec
total size is 16.37M  speedup is 1.06
#

Above command will automatically create a folder “rpms_db” under the /tmp folder in our local machine.

Example 6: Specify remote shell during synchronization (rsync -e ssh)

In rsync command we can specify the remote shell like ssh during synchronization, let’s assume we want to use secure communication between source to destination machine via rsync then we should use ssh as remote shell, example is shown below

Example 7: Display Synchronization progress in rsync command output

If you want to see the sync or copy progress in rsync command then use “–progress“, example is shown below

# rsync -avh --progress This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt/rpms_db /tmp
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
receiving incremental file list
……………………………………………………………………………………………………..
rpms_db/
rpms_db/httpd-2.4.6-88.el7.centos.x86_64.rpm
        2.84M 100%   35.22MB/s    0:00:00 (xfr#6, to-chk=18/25)
rpms_db/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm
        92.50K 100%    1.13MB/s    0:00:00 (xfr#7, to-chk=17/25)
rpms_db/postfix-2.10.1-7.el7.x86_64.rpm
        2.56M 100%   14.44MB/s    0:00:00 (xfr#17, to-chk=7/25)
rpms_db/samba-4.8.3-4.el7.x86_64.rpm
        696.47K 100%    3.71MB/s    0:00:00 (xfr#19, to-chk=5/25)
rpms_db/samba-client-libs-4.8.3-4.el7.x86_64.rpm
        5.07M 100%   19.90MB/s    0:00:00 (xfr#20, to-chk=4/25)
rpms_db/samba-common-4.8.3-4.el7.noarch.rpm
        210.98K 100%  844.42kB/s    0:00:00 (xfr#21, to-chk=3/25)
rpms_db/samba-common-libs-4.8.3-4.el7.x86_64.rpm
        167.51K 100%  667.70kB/s    0:00:00 (xfr#22, to-chk=2/25)
rpms_db/samba-common-tools-4.8.3-4.el7.x86_64.rpm
        458.38K 100%    1.77MB/s    0:00:00 (xfr#23, to-chk=1/25)
rpms_db/samba-libs-4.8.3-4.el7.x86_64.rpm
        282.33K 100%    1.09MB/s    0:00:00 (xfr#24, to-chk=0/25)
sent 484 bytes  received 16.38M bytes  3.64M bytes/sec
total size is 16.37M  speedup is 1.00
#

Example 8: Copy the directory structure without copying files

There are some scenarios where we want to copy the directory structure skiping files only from local machine to remote or vice versa.

Let’s suppose we want to copy the directory structure of “/home/docs” from local machine to remote machine (192.168.1.29) under /opt folder.

# rsync -av -f"+ */" -f"- *" /home/docs This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt/
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
building file list ... done
………………………………………………
docs/
sent 43 bytes  received 19 bytes  17.71 bytes/sec
total size is 0  speedup is 0.00
#

Example 9: Resume large file transfer after getting failed in scp

There are some scenarios in linux admin profile where we have started copying a larger file using scp command, but it got terminated in the middle and we can’t afford to start copying it again using scp because of its large size and time consumption.

So in this situation rsync command can used as it can start copying the file from where it left off or terminated, or in other words rsync can transfer the files which are partially copied using scp command. Example is shown below,

# scp This email address is being protected from spambots. You need JavaScript enabled to view it.:/root/ubuntu-18.04-desktop-amd64.iso /opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
ubuntu-18.04-desktop-amd64.iso                   28%  526MB  61.5MB/s   00:21 ETA
^CKilled by signal 2.
#
# rsync -P --rsh=ssh This email address is being protected from spambots. You need JavaScript enabled to view it.:/root/ubuntu-18.04-desktop-amd64.iso /opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
ubuntu-18.04-desktop-amd64.iso
1,921,843,200 100%   18.47MB/s    0:01:39 (xfr#1, to-chk=0/1)
#

Example 10: Delete files at destination if it is not present in source (–delete)

If you have already synced files from source to destination and from source you have deleted the files then you can force rsync command to delete the files on destination using the –delete option, example is shown below

# rsync -avz --delete /opt/rpms_db This email address is being protected from spambots. You need JavaScript enabled to view it.:/tmp/rpms_db
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
sending incremental file list
deleting rpms_db/apr-util-1.5.2-6.el7.x86_64.rpm
deleting rpms_db/apr-1.4.8-3.el7_4.1.x86_64.rpm
rpms_db/
sent 862 bytes  received 105 bytes  276.29 bytes/sec
total size is 15,947,152  speedup is 16,491.37
#

Example 11: Put limit on file transfer size (–max-size)

If you don’t want to transfer or copy the large files using rsync then use the option ‘–max-size={specify-size-here}’, let’s assume we don’t we don’t want to transfer the files whose size is more than 500K,

Note: To specify the size in MB use M and for GB use G.

# rsync -avz --max-size='500K' /opt/rpms_db This email address is being protected from spambots. You need JavaScript enabled to view it.:/tmp

Example 12: Do not sync/copy the modified file at destination

There are some scenarios where we modified the files at destination and we don’t want to overwrite the modified files from source while syncing, so this can be easily achieved using -u option in rsync command

# rsync -avzu This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt/tech /home/docs/tech
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
receiving incremental file list
sent 37 bytes  received 4,960 bytes  1,427.71 bytes/sec
total size is 37,771,649  speedup is 7,558.87
#

Example 13: Remove files from source after synchronization (–remove-source-files)

Let’s suppose you want to delete files from source once the synchronization is completed using rsync command. In the below example, folder from local system “/home/docs/tech.tgz” is synced to remote system (192.168.1.29), once the synchronization is completed, it will delete the file from source.

# rsync --remove-source-files -zvh /home/docs/tech.tgz This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
tech.tgz
sent 597 bytes  received 43 bytes  182.86 bytes/sec
total size is 518  speedup is 0.81
#
# ls -l /home/docs/tech.tgz
ls: cannot access /home/docs/tech.tgz: No such file or directory
#

Example 14: Dry run rsync command before actual synchronization

There can be some situations where we are not about behavior of rsync command so in such cases it is better to do dry run of rsync

# rsync --dry-run --remove-source-files -zvh /home/docs/projects.tar  This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
projects.tar
sent 51 bytes  received 19 bytes  7.37 bytes/sec
total size is 981.24M  speedup is 14,017,682.29 (DRY RUN)
#

Example 15: Include and Exclude files during synchronization with rsync

There are some situations where we want to copy or sync files & directories of specific type and want to exclude files or directories of specific type. Rsync command supports both include and exclude options.

In the below example want to copy the files of type pdf and rpm and exclude png file types.

# rsync -avz -e ssh --include '*.pdf *.rpm' --exclude '*.png' /home/docs/tech This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
sending incremental file list
tech/
tech/OpenStack-Networking.pdf
tech/httpd-2.4.6-88.el7.centos.x86_64.rpm
tech/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm
tech/postfix-2.10.1-7.el7.x86_64.rpm
sent 9,469,912 bytes  received 96 bytes  2,705,716.57 bytes/sec
total size is 11,647,907  speedup is 1.23
#

Example 16: Put restriction on data transfer speed in rsync (–bwlimit=<KB/s>)

In rsync command we can set the bandwidth limit for data transfer from one machine to another, use ‘–bwlimit=<KB/S>‘ option in rsync command to put limit on data transfer speed.

Let’s assume we want to set maximum data transfer rate (speed) is 600 KB/s with rsync, example is shown below

# rsync -avz --progress --bwlimit=600 /home/pkumar/tech This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
sending incremental file list
tech/
tech/OpenStack-Networking.pdf
6,153,239 100%  910.02kB/s    0:00:06 (xfr#1, to-chk=6/8)
tech/httpd-2.4.6-88.el7.centos.x86_64.rpm
2,844,028 100%  615.28kB/s    0:00:04 (xfr#2, to-chk=5/8)
tech/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm
92,504 100%  507.51kB/s    0:00:00 (xfr#3, to-chk=4/8)
tech/pkumar-2.png
0 100%    0.00kB/s    0:00:00 (xfr#4, to-chk=3/8)
tech/pkumar-3.png
0 100%    0.00kB/s    0:00:00 (xfr#5, to-chk=2/8)
tech/pkumar.png
0 100%    0.00kB/s    0:00:00 (xfr#6, to-chk=1/8)
tech/postfix-2.10.1-7.el7.x86_64.rpm
2,558,136 100%  594.80kB/s    0:00:04 (xfr#7, to-chk=0/8)
sent 9,470,087 bytes  received 153 bytes  485,653.33 bytes/sec
total size is 11,647,907  speedup is 1.23
#

Example 17: View the difference in files & directories between source and destination

Use “-i” option in rsync command to list the difference in files and directories between source and destination. Example is shown below

# rsync -avzi /home/pkumar/tech This email address is being protected from spambots. You need JavaScript enabled to view it.:/opt
This email address is being protected from spambots. You need JavaScript enabled to view it.'s password:
sending incremental file list
.d..t...... tech/
<f.st...... tech/projects.txt
sent 438 bytes  received 45 bytes  138.00 bytes/sec
total size is 11,648,064  speedup is 24,116.07
#

As per above command output, there is difference in file called “projects.txt” on destination. Following are the meaning of the keywords in above output,

d: indicates change in destination file
f: indicates a file
t: indicates change in timestamps
s: indicates change in size