I want to copy (rsync to a remote server) a directory tree whenever file uploaded or deleted in /var/www/html/upload/ directory under Linux operating systems for backup purpose and/or load balancing purpose without getting into complicated file sharing setups such as NFS or GFS OR iSCSI storage. How do I monitor /var/www/html/upload/ and its subdirectory for new files and executes rsync command to make copy back to www2.example.com:/var/www/html/upload/?

inotify is an inode-based filesystem notification technology. It provides the possibility to monitor various events on files in filesystems. It is a very much great replacement of (obsolete) dnotify. inotify brings a comfortable way how to manage files used in your applications. The incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to Linux or Unix cron jobs. This page shows how to set up incrond and sync files between two web server nodes.

How to install incrond

Type the following dnf command/yum command under RHEL / Fedora / CentOS Linux:
$ sudo yum install incron
Type the apt command/apt-get command under Debian / Ubuntu Linux:
$ sudo apt-get install incron
Linux install incrond - inotify cron (incron) daemon

Configuration Files

  • /etc/incron.conf – Main incron configuration file
  • /etc/incron.d/ – This directory is examined by incrond for system table files. You should put all your config file here as per directory or domain names.
  • /etc/incron.allow – This file contains users allowed to use incron.
  • /etc/incron.deny – This file contains users denied to use incron.
  • /var/spool/incron – This directory is examined by incrond for user table files which is set by users running the incrontab command.

incron Syntax

The syntax is as follows:

<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh
/sales IN_DELETE /root/scripts/sync.sh
/var/named/chroot/var/master IN_CREATE,IN_ATTRIB,IN_MODIFY /sbin/rndc reload

Where,

  • <directory> – It is nothing but path which is an absolute filesystem path such as /home/data. Any changes made to this path will result into command or action.
  • <file change mask> – Mask is is nothing but various file system events such as deleting a file. Each event can result into command execution. Use the following masks:
    • IN_ACCESS – File was accessed (read)
    • IN_ATTRIB – Metadata changed (permissions, timestamps, extended attributes, etc.)
    • IN_CLOSE_WRITE – File opened for writing was closed
    • IN_CLOSE_NOWRITE – File not opened for writing was closed
    • IN_CREATE – File/directory created in watched directory
    • IN_DELETE – File/directory deleted from watched directory
    • IN_DELETE_SELF – Watched file/directory was itself deleted
    • IN_MODIFY – File was modified
    • IN_MOVE_SELF – Watched file/directory was itself moved
    • IN_MOVED_FROM – File moved out of watched directory
    • IN_MOVED_TO – File moved into watched directory
    • IN_OPEN – File was opened
    • The IN_ALL_EVENTS symbol is defined as a bit mask of all of the above events.
  • <command or action> – Run command or scripts when mask matched on given directory.
  • options – It can be any one of the following with command (i.e. you can pass it as arg to your command):
  1. $$ – dollar sign
  2. $@ – watched filesystem path (see above)
  3. $# – event-related file name
  4. $% – event flags (textually)
  5. $& – event flags (numerically)

Turn Service On

Type the following command on a CentOS/RHEL:
# service incrond start
# chkconfig incrond on
###################################################
### systemd based Linux distro such as CentOS/RHEL 7.x/8.x, try ##
###################################################
# systemctl enable incrond.service
# systemctl start incrond.service

Linux incrond inotify monitor dir for changes and take action

Type the following command to edit your incrontab
incrontab -e
Run logger command when file created or deleted from /tmp directory:
/tmp IN_ALL_EVENTS logger "/tmp action for $# file"
Save and close the file. Now cd to /tmp and create a file:
$ cd /tmp
$ >foo
$ rm foo
To see message, enter:
$ sudo tail -f /var/log/messages
Sample outputs:
Jul 17 18:39:25 vivek-desktop logger: "/tmp action for foo file"

How Do I Run Rsync Command To Replicate Files For /var/www/html/upload Directory?

Type the following command:
# incrontab -e
Append the following command:
/var/www/html/upload/ IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
Now, wherever files are uploaded in /var/www/html/upload/ directory, rsync will be executed to sync files to www2.example.com server. Make sure ssh keys are set for password less login.

How Do I Monitor /var/www/html/upload/ and Its Subdirectories Recursively?

You cannot monitor /var/www/html/upload/ directory recursively. However, you can use the find command to add all sub-directories as follows:
find /var/www/html/upload -type d -print0 | xargs -0 -I{} echo "{} IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/" > /etc/incron.d/webroot.conf
This will create /etc/incron.d/webroot.conf config as follows:

/var/www/html/upload IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/css IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/1 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/js IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/3 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/11 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/12 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/files IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/images IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/

How Do I Troubleshoot Problems?

You need to see /var/log/cron log file:
# tail -f /var/log/cron
# grep something /var/log/cron

Further readings:

  • inotify project home page here
  • man pages – incrontab(5), incrond, and incron.conf

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Similar Posts