Saturday, June 13, 2015

Linux incrond inotify: Monitor Directories For Changes And Take Action

want to copy (rsync to remote server) a directory tree whenever file uploaded or deleted in /var/www/html/upload/ directory under Linux operating systems for backup purpose and/or load balancing purpose without getting into complex file sharing setup such as NFS or GFS iscsi storage. How do I monitor /var/www/html/upload/ and its subdirectory for new files and executes rsync command to make copy back to www2.example.com:/var/www/html/upload/?

inotify is an inode-based filesystem notification technology. It provides possibility to simply monitor various events on files in filesystems. It is a very much powerful replacement of (obsolete) dnotify. inotify brings a comfortable way how to manage files used in your applications.
The incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to cron.

Install incron

Type the following command under RHEL / Fedora / CentOS Linux:
$ sudo yum install incron
Type the following command under Debian / Ubuntu Linux:
$ sudo apt-get install incron

Configuration Files

  • /etc/incron.conf - Main incron configuration file
  • /etc/incron.d/ - This directory is examined by incrond for system table files. You should put all your config file here as per directory or domain names.
  • /etc/incron.allow - This file contains users allowed to use incron.
  • /etc/incron.deny - This file contains users denied to use incron.
  • /var/spool/incron - This directory is examined by incrond for user table files which is set by users running the incrontab command.

incron Syntax

The syntax is as follows:
<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh
/sales IN_DELETE /root/scripts/sync.sh
/var/named/chroot/var/master IN_CREATE,IN_ATTRIB,IN_MODIFY /sbin/rndc reload
Where,
  • <directory> - It is nothing but path which is an absolute filesystem path such as /home/data. Any changes made to this path will result into command or action.
  • <file change mask> - Mask is is nothing but various file system events such as deleting a file. Each event can result into command execution. Use the following masks:
    • IN_ACCESS - File was accessed (read)
    • IN_ATTRIB - Metadata changed (permissions, timestamps, extended attributes, etc.)
    • IN_CLOSE_WRITE - File opened for writing was closed
    • IN_CLOSE_NOWRITE - File not opened for writing was closed
    • IN_CREATE - File/directory created in watched directory
    • IN_DELETE - File/directory deleted from watched directory
    • IN_DELETE_SELF - Watched file/directory was itself deleted
    • IN_MODIFY - File was modified
    • IN_MOVE_SELF - Watched file/directory was itself moved
    • IN_MOVED_FROM - File moved out of watched directory
    • IN_MOVED_TO - File moved into watched directory
    • IN_OPEN - File was opened
    • The IN_ALL_EVENTS symbol is defined as a bit mask of all of the above events.
  • <command or action> - Run command or scripts when mask matched on given directory.
  • options - It can be any one of the following with command (i.e. you can pass it as arg to your command):
    1. $$ - dollar sign
    2. $@ - watched filesystem path (see above)
    3. $# - event-related file name
    4. $% - event flags (textually)
    5. $& - event flags (numerically)

Turn Service On

Type the following command:
# service incrond start
# chkconfig incrond on

Examples:

Type the following command to edit your incrontab
incrontab -e
Run logger command when file created or deleted from /tmp directory:
/tmp IN_ALL_EVENTS logger "/tmp action for $# file"
Save and close the file. Now cd to /tmp and create a file:
$ cd /tmp
$ >foo
$ rm foo

To see message, enter:
$ sudo tail -f /var/log/messages
Sample outputs:
Jul 17 18:39:25 vivek-desktop logger: "/tmp action for foo file"

How Do I Run Rsync Command To Replicate Files For /var/www/html/upload Directory?

Type the following command:
# incrontab -e
Append the following command:
/var/www/html/upload/ IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
Now, wherever files are uploaded in /var/www/html/upload/ directory, rsync will be executed to sync files to www2.example.com server. Make sure ssh keys are set for password less login.

How Do I Monitor /var/www/html/upload/ and Its Subdirectories Recursively?

You cannot monitor /var/www/html/upload/ directory recursively. However, you can use the find command to add all sub-directories as follows:
find /var/www/html/upload -type d -print0 | xargs -0 -I{} echo "{} IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/" > /etc/incron.d/webroot.conf
This will create /etc/incron.d/webroot.conf config as follows:
/var/www/html/upload IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/css IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/1 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/js IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/3 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/11 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/12 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/files IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/images IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/

How Do I Troubleshoot Problems?

You need to see /var/log/cron log file:
# tail -f /var/log/cron
# grep something /var/log/cron

Further readings:

No comments:

Post a Comment