1

I need a simple script for cent OS 5-7 to do the following:

I need to offload all tar.gz files from a dir to a share each server has access to and is mounted locally.

webserver1

/path/to/logs/

Files in this folder look like this:

server.log-compressed.tar.gz

I need to move everything in /path/to/logs to /nas/log-save/%hostname% -P

I would like the rsync to check the destination on the nas to ensure the move has completed then delete the tar.gz files it just moved to the nas from the local server.

I’m not sure how to accomplish this.

Thanks all!

6 Spice ups
2

I don’t know what " -P " means after “%hostname%”. (For that matter, I don’t know what “%hostname%” means.)

Test this:

rsync --archive /path/to/logs /nas/log-save/$(hostname -s) --dry-run

When you’re comfortable some variation of the above will work, test it without “–dry-run” from the command line.

When you’re comfortable that that works from the command line, test this:

rsync --archive /path/to/logs /nas/log-save/$(hostname -s) && mv *.tar.gz /some/other/dir/

&& won’t execute the move ( “mv” ) unless the rsync succeeded.

If that works you can script it.

I would not have the script do a removal. Instead, “mv” the files to a separate directory, then run a separate script out of cron that tests the access times of the mv’d files, and if older than however many days, remove them.

3

Here’s a copy of the one I have… see if this works for you:

Just have an external file to use for your list of servers/directories.

email_rcpt=email@yourplace.com
TIME=`date +"%y/%m/%d"`
TARRETVAL=-1

BACKUP_HOME=/backups/backup_store
targetlist=/home/rsyncer/backup_tar.conf
retention=30

#Cycle through each server and back up each directory

#Externalize backup target list using $targetlist file (backup_tar.conf).
for d in `cat ${targetlist}`; do
  SSHSRVRNAME=${d%:*}
  BKUPFOLDER=${d#*:}
  BKDIR="$BACKUP_HOME/$TIME/${SSHSRVRNAME}${BKUPFOLDER}"
  if [ ! -d "$BKDIR" ]; then mkdir -p $BKDIR; fi
  ERRLOG="$BACKUP_HOME/$TIME/$SSHSRVRNAME/errlog.log"
  touch $ERRLOG

  echo "Backup Error Log file: $SSHSRVRNAME" >> $ERRLOG
  echo "Backup target = " $SSHSRVRNAME:$BKUPFOLDER >> $ERRLOG
  echo "Backup destination = " $BKDIR >> $ERRLOG
  ssh `whoami`@$SSHSRVRNAME sudo tar cvzpPf - ${BKUPFOLDER%\/} 1>$BKDIR/backuptar.tgz 2>>$ERRLOG
  TARRETVAL=$?

  email_priority=5
  if [ $TARRETVAL == 0 ]; then
    email_subject="${d} Backup Successful"
    echo "Backup successful for server $d. The return code is $TARRETVAL." >> $ERRLOG
  else
    email_priority=1
    if [ $TARRETVAL == 2 ]; then
      email_subject="Fatal Error! Backup failed for server $d"
      echo "Backup failed for server $d. Fatal, unrecoverable error occurred. The return code is $TARRETVAL." >> $ERRLOG
    elif [ $TARRETVAL == 1 ]; then
      email_subject="Warning! Some files changed while being archived for server $d"
      echo "Some files differ for server $d. The resulting archive does not contain the exact copy of the file set. The return code is $TARRETVAL." >> $ERRLOG
    else
      email_subject="Backup failed for server $d"
      echo "Backup failed for server $d. The return code is $TARRETVAL." >> $ERRLOG
    fi
  fi
  tail -10 $ERRLOG|/usr/sbin/sendEmail -q -u "${email_subject}" -f `hostname`@yourplace.com -s mailrelay.yourplace.com:25 -o message-header="X-Priority: ${email_priority}" -t $email_rcpt
done

# Clean-up: Delete the backups that are two months old. Keep only the last 30 days backup in the backup server.
for i in `find $BACKUP_HOME/ -type f -mtime +${retention} -print`; do rm -rf $i; done
for i in `find $BACKUP_HOME/ -type d -empty -print`; do rmdir $i; done

exit 0
4

I have used the ‘remove source’ option in rsync. After transfer, it removes the source files. So, if it hangs, or the connection drops, nothing is deleted.

So, something like this:

rsync --remove-source-files -avz /path/to/src/ /path/to/dest

1 Spice up
5 
#!/bin/sh
curentdatea=$(date +%d_%m_%y-%R)

date

mkdir /filestore/log-save/$HOSTNAME -p

rsync --remove-source-files -bav --suffix='$currentdatea' --include '*.tar.gz' --exclude '*' /jboss/server/default/log/ /filestore/log-save/$HOSTNAME

This is what I did.