Obackup v1.84 RC3

CHANGELOG.md

@@ -1,17 +1,29 @@
-## FUTURE IMPROVEMENTS
+SHORT FUTURE IMPROVEMENTS
+-------------------------
 
-- Exit trap function must also stop child processes
 - Rewrite rsync exclude patterns using \"pattern\" instead of escaped chars
+- Clean most of recursive task creation code
 
-## Known issues
+FAR FUTURE IMPROVEMENTS
+-----------------------
 
-- Backup size check counts excluded patterns
+- (Secret world domination... Still need to get bald and get a cat)
-- Recursive task creation from directories does only include subdirectories, but no files in root directory
-- Bandwidth parameter is ignored by SQL backups.
-- Missing symlink support under MSYS
 
-## Latest changelog
+KNOWN ISSUES
+------------
 
+- Backup size check does not honor rsync exclude patterns
+- Bandwidth parameter is ignored for SQL backups
+- Missing symlink support when run from MSYS environment
+
+CHANGELOG
+---------
+
+- 02 Nov. 2013: v1.84 RC3
+- Updated documentation
+- Minor rewrites in recursive backup code
+- Added base directory files backup for recursive directories backup
+- Minor improvements on permission checks
 - Added local and remote OS detection
 - Fixed ping arguments for FreeBSD compatibility
 - Added MSYS (MinGW minimal system) bash compatibility under Windows
@@ -63,8 +75,5 @@
 - Updated command line argument --silent processing
 - Added remote before and after command execution hook
 - Added local before and after command execution hook
-
+- 14 Jun 2013
-14 Jun 2013
------------
-
 - Initial public release, fully functionnal

README.md

@@ -1,29 +1,26 @@
 obackup
 =======
 
-A small robust file & database backup script tailored for multiple virtualhost backups locally or remotely via ssh.
+A small robust file & database backup script for local to local or remote to local backups via ssh.
-Yet it is very versatile and actually works for a lot of backup tasks.
+Works especially well for multiple virtualhost backups with 'backup divide task' functionnality.
 
 ## About
 
 OBackup is designed from ground to make the backup process as reliable as possible.
 It divides the whole backup process into tasks, allowing each task to execute for a certain amount of time.
-If a task doesn't finish in time, it's stopped and the next task is processed.
+If a task doesn't finish in time, it's stopped and the next task in list is processed.
 Before a task gets stopped, a first warning message is generated telling the task takes too long.
-Every action gets logged, and at the end of the backup process, if there was a warning,
+Every action gets logged, and if a warning has been generated, a task gets stopped or an error happens, an alert email will be sent.
-a stopped task or an error an alert email will be sent.
 
 Remote backups are initiated from the backup server instead of the production server, so hacked servers won't get ssh access to the backup server.
 
-OBackup can enumerate and backup all MariaDB / MySQL databases present on a server
+OBackup can enumerate and backup all MariaDB / MySQL databases present on a server.
 It can also enumarate all subdirectories of a given path and process them as separate tasks (usefull for multiple vhosts).
-It will do several checks before launching a backup like execution checks, dryruns,
+It will do several checks before launching a backup like execution checks, dryruns, checking backup size and available local disk space.
-checking backup size and available local disk space.
 
-Obackup will work well to backup to a snapshot aware filesystem like ZFS or btrfs.
+Obackup can execute local and remote commands before and after backup execution,
-In case you don't work with one of these, it may also rotate backups for you.
+thus providing an easy way to handle snapshots (see https://github.com/deajan/zsnap for a zfs snapshot management script).
-
+It may also rotate backups for you.
-Obackup can execute local and remote commands before and after backup execution, thus providing possibilites to handle snapshots (see https://github.com/deajan/zsnap for a zfs snapshot management script).
 
 As of today, obackup has been tested successfully on RHEL / CentOS 5, CentOS 6, Debian 6.0.7 and Linux Mint 14.
 Currently, Obackup also runs on FreeBSD and Windows MSYS environment, altough it is not fully tested yet.
@@ -33,41 +30,52 @@ Feel free to drop me a mail for limited support in my free time.
 ## Installation
 
 You can download the latest obackup script from authors website.
-You may also clone this git which will maybe have some more recent build.
+You may also clone the following git which will maybe have some more recent builds.
 
     $ git clone git://github.com/deajan/obackup.git
     $ chmod +x ./obackup.sh
 
 Obackup needs to run with bash shell, using any other shell will most probably fail.  
-Once you have grabbed a copy of Obackup, just edit the config file with your favorite text editor to setup your environment and you're ready to run. A detailled documentation can be found in the DOCUMENTATION.md file.
+Once you have grabbed a copy, just edit the config file with your favorite text editor to setup your environment and you're ready to run.
-You can run multiple instances of obackup scripts with different backup environments. Just create another configuration file, edit it's environment and you're ready to run concurrently.
+A detailled documentation can be found on the author's site.
+You can run multiple instances of obackup scripts with different backup environments. Just create another configuration file,
+edit it's environment and you're ready to run concurrently.
 
 Performing remote backups requires you to create RSA private / public key pair. Please see documentation for further information.
 Performing mysql backups requires to create a mysql user with SELECT rights on all databases. Please see documentation for further information.
-Performing backup with SUDO_EXEC option requires to configure sudoers file to allow some commands without password. Keep in mind that running backup as superuser might be a security risk. You should generally prefer a read only enabled user. Please see documentation for further information.
+Performing backup with SUDO_EXEC option requires to configure sudoers file to allow some commands without password.
+Keep in mind that running backup as superuser might be a security risk. You should generally prefer a read only enabled user.
+Please see documentation for further information.
 
 ## Usage
 
 MariaDB / MySQL backups are consistent because dumps are done with the --single-transaction option.
-File backups can be done directly if data won't change while a backup is going on (generally true on vhosts), but backing up a snapshot of the actual data is preferable as it will stay consistent. LVM, zfs or btrfs snapshots will do fine.
+File backups can be done directly if data won't change while a backup is going on (generally true on vhosts),
+but backing up a snapshot of the actual data is preferable as it will stay consistent. LVM, zfs or btrfs snapshots will do fine.
 
-You may try your setup by specifying the "--dry" parameter which will run a simulation of what will be done.
+You may try your setup by specifying the "--dry" parameter which will run a simulation of what will be done. Specifying "--verbose" will also list any command
+that's actually launched and it's result.
 
     $ ./obackup.sh path/to/backup.conf --dry
     $ ./obackup.sh path/to/backup.conf
+    $ ./obackup.sh path/to/backup.conf --silent
+	
 
 One you're happy with a test run, you may run obackup as a cron task with the "--silent" parameter so output will not be written to stdout.
-All backup activity is logged to "/var/log/obackup_backupname.log".
+All backup activity is logged to "/var/log/obackup_backupname.log" or current directory if /var/log is not writable.
 
 ## Final words
 
-Backup tasks aren't always reliable, connectivity loss, insufficient disk space, hacked computers with tons of mangas to backup... Anything can happen. Obackup will sent your a warning email for every issue it can handle.
+Backup tasks aren't always reliable, connectivity loss, insufficient disk space, hacked servers with tons of unusefull stuff to backup... Anything can happen.
-Nevertheless, you should assure yourself that your backup tasks will get done the way you meant it. Also, a backup isn't valuable until you're sure it's restoration will be a success. Try to restore your backups to check whether everything is okay. Backups will keep file permissions and owners, but may loose ACLs if destination file system won't handle them. 
+Obackup will sent your a warning email for every issue it can handle.
+Nevertheless, you should assure yourself that your backup tasks will get done the way you meant it. Also, a backup isn't valuable until you're sure
+you can successfully restore. Try to restore your backups to check whether everything is okay. Backups will keep file permissions and owners,
+but may loose ACLs if destination file system won't handle them. 
 
 ## Author
 
-Orsiris "Ozy" de Jong.
+Feel free to mail me for limited support in my free time :)
-ozy@badministrateur.com
+Orsiris "Ozy" de Jong | ozy@netpower.fr
 
 
  

host_backup.conf

@@ -1,119 +1,163 @@
 #!/bin/bash
 
 ###### Remote (or local) backup script for files & databases
-###### (L) 2013 by Ozy de Jong (www.badministrateur.com)
+###### (L) 2013 by Ozy de Jong (www.netpower.fr)
-###### Config file rev 2109201301
+###### Config file rev 0211201301
 
-## Backup identification, any string you want 
+## ---------- GENERAL BACKUP OPTIONS
+
+## Backup identification name. 
 BACKUP_ID="your backup name"
 
-## Leaving this empty will create a logfile at /var/log/obackup_version_BACKUP_ID.log (or current directory if /var/log doesn't exist)
+## Log file location. Leaving this empty will create log file at /var/log/obackup_version_BACKUP_ID.log (or current directory if /var/log doesn't exist)
 LOGFILE=""
 
-## General backup options
+## Backup databases
 BACKUP_SQL=no
+## Backup files
 BACKUP_FILES=yes
 
-## Local storage paths
+## ---------- LOCAL BACKUP STORAGE OPTIONS
+
+## Local storage paths where to put backups
 LOCAL_SQL_STORAGE="/home/storage/backup/sql"
 LOCAL_FILE_STORAGE="/home/storage/backup/files"
-## Create sync directories if they do not exist
+
-CREATE_DIRS=no
+## Create backup directories if they do not exist
-## Keep the absolute source path in your backup, eg: /your/backup/storage/the/remote/server/files
+CREATE_DIRS=yes
-## You should leave this enabled if you use recursive directories backup lists or they'll all end in the same path.
+
+## Keep absolute source path in your backup, eg: /your/backup/storage/the/remote/server/files
+## You should leave this enabled if you intend to use 'backup task division' functionality of OBackup, or everything will end up in the same directory.
 LOCAL_STORAGE_KEEP_ABSOLUTE_PATHS=yes
-## Generate an alert if backup size is lower than given value in Kb, useful for local mounted backups.
+
+## Generate an alert if backup size is lower than given value in Kb (this can also help identifying empty mount dirs)
 BACKUP_SIZE_MINIMUM=1024
+
 ## Generate an alert if local storage free space is lower than given value in Kb.
 LOCAL_STORAGE_WARN_MIN_SPACE=1048576
 
-## Bandwidth limit Kbytes / second. Leave 0 to disable limitation.
+## ---------- MISC OPTIONS
+
+## Bandwidth limit Kbytes / second for file backups. Leave 0 to disable limitation.
 BANDWIDTH=0
 
-## If enabled, file backups will be processed with sudo command. See documentation for /etc/sudoers configuration ("find", "du" and "rsync" need to be allowed). Requiretty needs to be disabled.
+## If enabled, file backups will be processed as superuser. See documentation for /etc/sudoers configuration ("find", "du" and "rsync" need to be allowed). Requiretty needs to be disabled.
 SUDO_EXEC=no
-## Paranoia option. Don't change this unless you read the documentation and still feel concerned about security issues.
+## Paranoia option. Don't change this unless you read the documentation.
 RSYNC_EXECUTABLE=rsync
 
-## Remote options (will make backups of remote system through ssh tunnel, public RSA key need to be put into /home/.ssh/authorized_keys in remote users home directory)
+## ---------- REMOTE BACKUP OPTIONS
-REMOTE_BACKUP=yes
+
+## The following options allow this Obackup instance to connect to a remote system via an ssh tunnel.
+## Needs public RSA key need to be put into ~/.ssh/authorized_keys in remote users home directory.
+REMOTE_BACKUP=no
 SSH_RSA_PRIVATE_KEY=~/.ssh/id_rsa
 REMOTE_USER=backupuser
 REMOTE_HOST=yourhost.local
 REMOTE_PORT=22
 ## ssh compression should be used unless your remote connection is good enough (LAN)
 SSH_COMPRESSION=yes
+## Remote rsync executable path. Leave this empty in most cases
+REMOTE_RSYNC_PATH=""
+
 ## Check for connectivity to remote host before launching remote backup tasks. Be sure the hosts responds to ping. Failing to ping will skip current task.
 REMOTE_HOST_PING=yes
 ## Check for internet access by pinging one or more 3rd party hosts before remote backup tasks. Leave empty if you don't want this check to be be performed. Failing to ping will skip current task.
 REMOTE_3RD_PARTY_HOSTS="www.kernel.org www.google.com"
-## Remote rsync executable path. Leave this empty in most cases
-REMOTE_RSYNC_PATH=""
 
-## Databases options
+## ---------- DATABASE BACKUP OPTIONS
+
+## Database backup user
 SQL_USER=backupuser
-## Save all databases except the ones specified in the exlude list. Every found database will be backed up as separate task (see documentation for explanation about tasks)
+
+## Enabling the following option will save all databases on local or remote given SQL instance except the ones specified in the exlude list.
+## Every found database will be backed up as separate backup task.
 DATABASES_ALL=yes
 DATABASES_ALL_EXCLUDE_LIST="test;mysql"
-# Alternatively, you can specifiy a manual list of databases to backup separated by spaces
+## Alternatively, if DATABASES_ALL=no, you can specifiy a list of databases to backup separated by spaces.
 DATABASES_LIST=""
-## Max backup execution time per DB task. Soft is warning only. Hard is warning, stopping backup task and processing next one. Time is specified in seconds
+
+## Max backup execution time per Database task. Soft max exec time generates a warning only. Hard max ecec time generates a warning and stops current backup task.
+## If a task gets stopped, next one in the task list gets executed. Time is specified in seconds.
 SOFT_MAX_EXEC_TIME_DB_TASK=3600
 HARD_MAX_EXEC_TIME_DB_TASK=7200
-## Preferred sql dump compression. Can be set to xz, lzma or gzip.
+
+## Preferred SQL dump compression. Can be set to xz, lzma or gzip.
 ## Generally, xz level 5 is a good compromise between cpu, memory hunger and compress ratio. Gzipped files are set to be rsyncable.
 COMPRESSION_PROGRAM=xz
 COMPRESSION_LEVEL=3
-## Dump compression should be done on remote side but can also be done locally to lower remote system usage (will take more bandwidth, check for ssh compression)
+
+## SQL Dump compression should be done on remote side but can also be done locally to lower remote system usage (will take more bandwidth, check for ssh compression)
 COMPRESSION_REMOTE=yes
 
-## Path separator. You can set whatever seperator you want in your directories list below. You may change this in case you have some esoteric filenames (try to use unconventional separators like | ).
+## ---------- FILES BACKUP OPTIONS
-PATH_SEPARATOR_CHAR=";"
+
-## File backup lists. Double quoted directory list separated by the $PATH_SEPARATOR_CHAR. Every directory will be processed as task (see documentation for explanation about tasks)
+## Directories backup list. List of semicolon separated directories that will be backed up recursively. Every directory will be processed as one backup task.
-DIRECTORIES_SIMPLE_LIST="/var/named"
+DIRECTORIES_SIMPLE_LIST="/var/named;/var/lib"
-## Recurse directory list separated by the $PATH_SEPARATOR_CHAR. Will create a backup task per subdirectory (one level only), eg RECURSE_LIST="/home /var" will create tasks "/home/dir1", "/home/dir2", ... "/home/dirN", "/var/log", "/var/lib"... "/var/whatever"
+
+## There's a special backup schema in Obackup called 'backup task division'  which creates one backup task per level 1 subdirectory of a directory.
+## This is VERY usefull to backup multiple virtualhosts as separate tasks without having to specifiy each one separately.
+## This may also be usefull dividing big data directories in subdirectories tasks.
+
+## Directories backup task division backup: Semicolon separated directories of which every level 1 subdirectory will be backed up recursively as a spearate backup task.
+## Example: "/home;/var" will create tasks "/home/dir1", "/home/dir2", ... "/home/dirN", "/var/log", "/var/lib"... "/var/whatever"
 DIRECTORIES_RECURSE_LIST="/home"
-## You can optionally exclude directories from RECURSE_LIST tasks, eg on the above example you could exclude /home/dir2 by adding it to RECURSE_EXCLUDE_LIST
+## You may optionally exclude subdirectories from task division. On the above example you could exclude /home/dir2 by adding it to DIRECTORIES_RECURSE_EXCLUDE_LIST
 DIRECTORIES_RECURSE_EXCLUDE_LIST="/home/backupuser;/home/lost+found"
-## Be aware that every recurse list will have it's own root (exclude pattern is relative from /home/web for /home/web/{recursedir})
+
+## Rsync exclude patterns, used by simple and division lists
 RSYNC_EXCLUDE_PATTERN="*/tmp;*/ftp/www/cache/cachefs;*/sessions"
-## Preserve ACLS. Make sure target FS can hold ACLs or you'll get loads of errors.
+
+## List separator char. You may set an alternative seperator char for your directories lists above.
+PATH_SEPARATOR_CHAR=";"
+
+## Preserve ACLS. Make sure source and target FS can hold same ACLs or you'll get loads of errors.
 PRESERVE_ACL=no
-## Preserve Xattr
+## Preserve Xattr. MAke sure source and target FS can hold same Xattr or you'll get loads of errors.
 PRESERVE_XATTR=no
-## Let RSYNC compress file transfers. Do not use if you already enabled SSH compression.
+
-RSYNC_COMPRESS=yes
+## Let RSYNC compress file transfers. Do not use this on local-local backup schemes. Also, this is not usefull if SSH compression is enabled.
+RSYNC_COMPRESS=no
+
 ## Max execution time per file backup task. Soft is warning only. Hard is warning, stopping backup and processing next one one file list. Tilme is specified in seconds
 SOFT_MAX_EXEC_TIME_FILE_TASK=3600
 HARD_MAX_EXEC_TIME_FILE_TASK=7200
 
+## ---------- ALERT OPTIONS
+
 ## Alert email adresses separated by a space character
 DESTINATION_MAILS="your@mail.address"
 
-## Windows only mail options (used by sendemail.exe)
+## Windows (MSYS environment) only mail options (used by sendemail.exe)
 SENDER_MAIL="alert@your.system"
 SMTP_SERVER=smtp.your.isp.com
 SMTP_USER=
 SMTP_PASSWORD=
 
-## Max execution time of whole backup process. Soft is warning only. Hard is warning and stopping whole backup process.
+## ---------- GENERAL BACKUP OPTIONS
+
+## Max execution time of whole backup process. Soft max exec time generates a warning only.
+## Hard max exec time generates a warning and stops the whole backup execution.
 SOFT_MAX_EXEC_TIME_TOTAL=30000
 HARD_MAX_EXEC_TIME_TOTAL=36000
 
-## Backup Rotation in case you don't use a snapshot aware file system like zfs or btrfs to perform a snapshot before every backup
+## Backup Rotation. You may rotate backups if you don't use snapshots on your backup server.
 ROTATE_BACKUPS=no
 ROTATE_COPIES=7
 
-## Commands that will be run  before and / or after backup execution (remote execution will only happen if REMOTE_BACKUP is set to yes). Very usefull to initiate snapshots.
+## ---------- EXECUTION HOOKS
-## Set max execution time to 0 if you want these commands not to get stopped, else set a value in seconds after which execution will be stopped.
+
+## Commands can will be run before and / or after backup execution (remote execution will only happen if REMOTE_BACKUP is set).
+## This is usefull to make a snapshot before backing up data, or even handle snapshots of backed up data.
 LOCAL_RUN_BEFORE_CMD=""
 LOCAL_RUN_AFTER_CMD=""
 
 REMOTE_RUN_BEFORE_CMD=""
 REMOTE_RUN_AFTER_CMD=""
 
+## Max execution time of commands before they get force killed. Leave 0 if you don't wan't this to happen. Time is specified in seconds.
 MAX_EXEC_TIME_PER_CMD_BEFORE=0
 MAX_EXEC_TIME_PER_CMD_AFTER=0
 
-## Stops obackup execution if one of the above commands fail
+## Stops whole backup execution if one of the above commands fail
 STOP_ON_CMD_ERROR=no

obackup.sh

@@ -2,8 +2,8 @@
 
 ###### Remote (or local) backup script for files & databases
 ###### (L) 2013 by Orsiris "Ozy" de Jong (www.netpower.fr)
-OBACKUP_VERSION=1.84preRC3-MSYS-compatible
+OBACKUP_VERSION=1.84RC3
-OBACKUP_BUILD=1110201301
+OBACKUP_BUILD=0211201302
 
 DEBUG=no
 SCRIPT_PID=$$
@@ -12,7 +12,7 @@ LOCAL_USER=$(whoami)
 LOCAL_HOST=$(hostname)
 
 ## Default log file until config file is loaded
-if [ -d /var/log ]
+if [ -w /var/log ]
 then
 	LOG_FILE=/var/log/obackup.log
 else
@@ -20,13 +20,13 @@ else
 fi
 
 ## Default directory where to store run files
-if [ -d /dev/shm ]
+if [ -w /dev/shm ]
 then
 	RUN_DIR=/dev/shm
-elif [ -d /tmp ]
+elif [ -w /tmp ]
 then
 	RUN_DIR=/tmp
-elif [ -d /var/tmp ]
+elif [ -w /var/tmp ]
 then
 	RUN_DIR=/var/tmp
 else
@@ -45,6 +45,7 @@ DIRECTORIES_EXCLUDED_LIST=""				# Processed list of recursive directorires that
 DIRECTORIES_TO_BACKUP=""				# Processed list of all directories to backup
 TOTAL_FILES_SIZE=0					# Total file size of $DIRECTORIES_TO_BACKUP
 
+# $RUN_DIR/obackup_remote_os_$SCRIPT_PID		Result of remote OS detection
 # $RUN_DIR/obackup_dblist_$SCRIPT_PID                   Databases list and sizes
 # $RUN_DIR/obackup_dirs_recurse_list_$SCRIPT_PID	Recursive directories list
 # $RUN_DIR/obackup_local_sql_storage_$SCRIPT_PID	Local free space for sql backup
@@ -57,7 +58,7 @@ TOTAL_FILES_SIZE=0					# Total file size of $DIRECTORIES_TO_BACKUP
 
 function Log
 {
-	echo "TIME: $SECONDS - $1" >> "$LOG_FILE"
+	echo -e "TIME: $SECONDS - $1" >> "$LOG_FILE"
 	if [ $silent -eq 0 ]
 	then
 		echo -e "TIME: $SECONDS - $1"
@@ -93,7 +94,7 @@ function TrapQuit
 	if type -p pkill > /dev/null 2>&1
 	then
 		pkill -TERM -P $$
-	elif [ "$LOCAL_OS" == "msys" ]
+	elif [ "$LOCAL_OS" == "msys" ] || [ "$OSTYPE" == "msys" ]
 	then
 		## This is not really a clean way to get child process pids, especially the tail -n +2 which resolves a strange char apparition in msys bash
 		for pid in $(ps -a | awk '{$1=$1}$1' | awk '{print $1" "$2}' | grep " $$$" | awk '{print $1}' | tail -n +2)
@@ -189,10 +190,10 @@ function CleanUp
 
 function SendAlert
 {
-	cat "$LOG_FILE" | gzip -9 > /tmp/obackup_lastlog.gz
+	cat "$LOG_FILE" | gzip -9 > $RUN_DIR/obackup_lastlog.gz
         if type -p mutt > /dev/null 2>&1
         then
-            echo $MAIL_ALERT_MSG | $(type -p mutt) -x -s "Backup alert for $BACKUP_ID" $DESTINATION_MAILS -a /tmp/obackup_lastlog.gz
+            echo $MAIL_ALERT_MSG | $(type -p mutt) -x -s "Backup alert for $BACKUP_ID" $DESTINATION_MAILS -a $RUN_DIR/obackup_lastlog.gz
             if [ $? != 0 ]
             then
                 Log "WARNING: Cannot send alert email via $(type -p mutt) !!!"
@@ -201,7 +202,7 @@ function SendAlert
             fi
         elif type -p mail > /dev/null 2>&1
 		then
-			echo $MAIL_ALERT_MSG | $(type -p mail) -a /tmp/obackup_lastlog.gz -s "Backup alert for $BACKUP_ID" $DESTINATION_MAILS
+			echo $MAIL_ALERT_MSG | $(type -p mail) -a $RUN_DIR/obackup_lastlog.gz -s "Backup alert for $BACKUP_ID" $DESTINATION_MAILS
             if [ $? != 0 ]
             then
                 Log "WARNING: Cannot send alert email via $(type -p mail) with attachments !!!"
@@ -226,7 +227,9 @@ function SendAlert
 			fi 
 		else
 		Log "WARNING: Cannot send alert email (no mutt / mail present) !!!"
+		rm -f $RUN_DIR/obackup_lastlog.gz
 		return 1
+		rm -f $RUN_DIR/obackup_lastlog.gz
 	fi
 }
 
@@ -287,11 +290,17 @@ function CheckEnvironment
 function GetOperatingSystem
 {
         LOCAL_OS_VAR=$(uname -spio)
-        if [ "$REMOTE_SYNC" == "yes" ]
+
+        if [ "$REMOTE_BACKUP" == "yes" ]
         then
-                eval "$SSH_CMD uname -spio > $RUN_DIR/obackup_remote_os_$SCRIPT_PID 2>&1 &"
+                eval "$SSH_CMD \"uname -spio\" > $RUN_DIR/obackup_remote_os_$SCRIPT_PID 2>&1"
-                REMOTE_OS_VAR=$(cat $RUN_DIR/obackup_remote_os_$SCRIPT_PID)
+		if [ $? != 0 ]
-        fi
+		then
+			LogError "Cannot Get remote OS type."
+		else
+                	REMOTE_OS_VAR=$(cat $RUN_DIR/obackup_remote_os_$SCRIPT_PID)
+		fi
+	fi
 
         case $LOCAL_OS_VAR in
                 "Linux"*)
@@ -325,6 +334,15 @@ function GetOperatingSystem
                 LogError "Running on remote >> $REMOTE_OS_VAR << not supported. Please report to the author."
                 exit 1
         esac
+
+	if [ "$DEBUG" == "yes" ]
+	then
+		Log "Local OS: [$LOCAL_OS_VAR]."
+		if [ "$REMOTE_BACKUP" == "yes" ]
+		then
+			Log "Remote OS: [$REMOTE_OS_VAR]."
+		fi
+	fi
 }
 
 # Waits for pid $1 to complete. Will log an alert if $2 seconds exec time exceeded unless $2 equals 0. Will stop task and log alert if $3 seconds exec time exceeded.
@@ -386,15 +404,16 @@ function RunLocalCommand
 		Log "Dryrun: Local command [$1] not run."
 		return 1
 	fi
+	Log "Running command [$1] on local host."
 	$1 > $RUN_DIR/obackup_run_local_$SCRIPT_PID 2>&1 &
 	child_pid=$!
 	WaitForTaskCompletion $child_pid 0 $2
 	retval=$?
 	if [ $retval -eq 0 ]
 	then
-		Log "Running command [$1] on local host succeded."
+		Log "Command succeded."
 	else
-		LogError "Running command [$1] on local host failed."
+		LogError "Command failed."
 	fi
 	
 	if [ $verbose -eq 1 ]
@@ -402,7 +421,7 @@ function RunLocalCommand
 		Log "Command output:\n$(cat $RUN_DIR/obackup_run_local_$SCRIPT_PID)"
 	fi
 	
-	if [ "$STOP_ON_CMD_ERROR" == "yes" ]
+	if [ "$STOP_ON_CMD_ERROR" == "yes" ] && [ $retval -ne 0 ]
         then
                 exit 1
         fi
@@ -419,15 +438,16 @@ function RunRemoteCommand
                 Log "Dryrun: Remote command [$1] not run."
                 return 1
         fi
+	Log "Running command [$1] on remote host."
 	eval "$SSH_CMD \"$1\" > $RUN_DIR/obackup_run_remote_$SCRIPT_PID 2>&1 &"
 	child_pid=$!
 	WaitForTaskCompletion $child_pid 0 $2
 	retval=$?
 	if [ $retval -eq 0 ]
 	then
-		Log "Running command [$1] succeded."
+		Log "Command succeded."
 	else
-		LogError "Running command [$1] failed."
+		LogError "Command failed."
 	fi
 	
 	if [ -f $RUN_DIR/obackup_run_remote_$SCRIPT_PID ] && [ $verbose -eq 1 ]
@@ -435,9 +455,9 @@ function RunRemoteCommand
 		Log "Command output:\n$(cat $RUN_DIR/obackup_run_remote_$SCRIPT_PID)"
 	fi
 
-        if [ "$STOP_ON_CMD_ERROR" == "yes" ]
+        if [ "$STOP_ON_CMD_ERROR" == "yes" ] && [ $retval -ne 0 ]
         then
-                exit 1
+		exit 1
         fi
 }
 
@@ -484,7 +504,7 @@ function CheckSpaceRequirements
 {
 	if [ "$BACKUP_SQL" != "no" ]
 	then
-		if [ -d $LOCAL_SQL_STORAGE ]
+		if [ -w $LOCAL_SQL_STORAGE ]
 		then
 			# Not elegant solution to make df silent on errors
 			df -P $LOCAL_SQL_STORAGE > $RUN_DIR/obackup_local_sql_storage_$SCRIPT_PID 2>&1
@@ -505,7 +525,7 @@ function CheckSpaceRequirements
 			fi
 		else
 			LOCAL_SQL_SPACE=0
-			LogError "SQL storage path [$LOCAL_SQL_STORAGE] doesn't exist."
+			LogError "SQL storage path [$LOCAL_SQL_STORAGE] doesn't exist or cannot write to it."
 		fi
 	else
 		LOCAL_SQL_SPACE=0
@@ -513,7 +533,7 @@ function CheckSpaceRequirements
 
         if [ "$BACKUP_FILES" != "no" ]
         then
-                if [ -d $LOCAL_FILE_STORAGE ]
+                if [ -w $LOCAL_FILE_STORAGE ]
                 then
                         df -P $LOCAL_FILE_STORAGE > $RUN_DIR/obackup_local_file_storage_$SCRIPT_PID 2>&1
                         if [ $? != 0 ]
@@ -533,7 +553,7 @@ function CheckSpaceRequirements
 			fi
                 else
 			LOCAL_FILE_SPACE=0
-                        LogError "File storage path [$LOCAL_FILE_STORAGE] doesn't exist."
+                        LogError "File storage path [$LOCAL_FILE_STORAGE] doesn't exist or cannot write to it."
                 fi
         else
         	LOCAL_FILE_SPACE=0
@@ -692,7 +712,12 @@ function ListDatabases
 		fi
 	done
 	IFS=$OLD_IFS
-	return 0
+
+	if [ $verbose -eq 1 ]
+	then
+		Log "Database backup list: $DATABASES_TO_BACKUP"
+		Log "Database exclude list: $DATABASES_EXCLUDED_LIST"
+	fi
 }
 
 function BackupDatabase
@@ -706,7 +731,7 @@ function BackupDatabase
 			LogError "Connectivity test failed. Stopping current task."
 			exit 1
 		fi
-		eval "$SSH_CMD mysqldump -u $SQL_USER --skip-lock-tables --single-transaction --database $1 | $COMPRESSION_PROGRAM -$COMPRESSION_LEVEL $COMPRESSION_OPTIONS > $LOCAL_SQL_STORAGE/$1.sql$COMPRESSION_EXTENSION"
+		sql_cmd="$SSH_CMD mysqldump -u $SQL_USER --skip-lock-tables --single-transaction --database $1 | $COMPRESSION_PROGRAM -$COMPRESSION_LEVEL $COMPRESSION_OPTIONS > $LOCAL_SQL_STORAGE/$1.sql$COMPRESSION_EXTENSION"
 	elif [ "$REMOTE_BACKUP" == "yes" ] && [ "$COMPRESSION_REMOTE" == "yes" ]
 	then
 		CheckConnectivityRemoteHost
@@ -715,10 +740,16 @@ function BackupDatabase
 			LogError "Connectivity test failed. Stopping current task."
 			exit 1
 		fi
-		eval "$SSH_CMD \"mysqldump -u $SQL_USER --skip-lock-tables --single-transaction --database $1 | $COMPRESSION_PROGRAM -$COMPRESSION_LEVEL $COMPRESSION_OPTIONS\" > $LOCAL_SQL_STORAGE/$1.sql$COMPRESSION_EXTENSION"
+		sql_cmd="$SSH_CMD \"mysqldump -u $SQL_USER --skip-lock-tables --single-transaction --database $1 | $COMPRESSION_PROGRAM -$COMPRESSION_LEVEL $COMPRESSION_OPTIONS\" > $LOCAL_SQL_STORAGE/$1.sql$COMPRESSION_EXTENSION"
 	else
-		mysqldump -u $SQL_USER --skip-lock-tables --single-transaction --database $1 | $COMPRESSION_PROGRAM -$COMPRESSION_LEVEL $COMPRESSION_OPTIONS > $LOCAL_SQL_STORAGE/$1.sql$COMPRESSION_EXTENSION
+		sql_cmd="mysqldump -u $SQL_USER --skip-lock-tables --single-transaction --database $1 | $COMPRESSION_PROGRAM -$COMPRESSION_LEVEL $COMPRESSION_OPTIONS > $LOCAL_SQL_STORAGE/$1.sql$COMPRESSION_EXTENSION"
 	fi
+
+	if [ $verbose -eq 1 ]
+        then
+                Log "SQL_CMD: $sql_cmd"
+        fi
+	eval $sql_cmd
 	exit $?
 }
 
@@ -780,14 +811,11 @@ function ListDirectories
 				LogError "Command output:\n$(cat $RUN_DIR/obackup_dirs_recurse_list_$SCRIPT_PID)"
 			fi
 			return 1
-		else
-			Log "Listing of recursive directories succeeded for $dir."
 		fi
 
 		OLD_IFS=$IFS
 		IFS=$' \n'
 		for line in $(cat $RUN_DIR/obackup_dirs_recurse_list_$SCRIPT_PID)
-#		while read line
 		do
 			file_exclude=0
 			for k in $DIRECTORIES_RECURSE_EXCLUDE_LIST
@@ -809,8 +837,12 @@ function ListDirectories
 			else
 				DIRECTORIES_EXCLUDED_LIST="$DIRECTORIES_EXCLUDED_LIST$PATH_SEPARATOR_CHAR'$line'"
 			fi
-#		done < <(cat $RUN_DIR/obackup_dirs_recurse_list_$SCRIPT_PID)
 		done
+		Log "Listing of recursive directories succeeded for $dir"
+		if [ $verbose -eq 1 ]
+		then
+			Log "\n$(cat $RUN_DIR/obackup_dirs_recurse_list_$SCRIPT_PID)"
+		fi
 		IFS=$OLD_IFS
 	done
 	DIRECTORIES_TO_BACKUP_RECURSE=$DIRECTORIES_TO_BACKUP
@@ -887,14 +919,15 @@ function Rsync
 	then
 		local_file_storage_path="$(dirname $LOCAL_FILE_STORAGE$i)"
 	else
-	#### Leave the last directory path if recursive task when absolute paths not set so paths won't be mixed up
+		local_file_storage_path=$LOCAL_FILE_STORAGE
-		if [ "$2" == "recurse" ]
-		then
-			local_file_storage_path="$LOCAL_FILE_STORAGE/$(basename $(dirname $i))"
-		else
-			local_file_storage_path="$LOCAL_FILE_STORAGE"
-		fi
 	fi
+
+	## Manage to backup recursive directories lists files only (not recursing into subdirectories)
+        if [ "$2" == "no-recurse" ]
+        then
+		RSYNC_EXCLUDE=$RSYNC_EXCLUDE" --exclude=*/*/"
+        fi
+
 	if [ ! -d $local_file_storage_path ]
 	then
 		mkdir -p "$local_file_storage_path"
@@ -909,9 +942,9 @@ function Rsync
 			LogError "Connectivity test failed. Stopping current task."
 			exit 1
 		fi
-		rsync_cmd="$(type -p $RSYNC_EXECUTABLE) $RSYNC_ARGS --stats --delete $RSYNC_EXCLUDE --rsync-path=\"$RSYNC_PATH\" -e \"$RSYNC_SSH_CMD\" \"$REMOTE_USER@$REMOTE_HOST:$1\" \"$local_file_storage_path\" > $RUN_DIR/obackup_rsync_output_$SCRIPT_PID 2>&1"
+		rsync_cmd="$(type -p $RSYNC_EXECUTABLE) $RSYNC_ARGS --stats $RSYNC_DELETE $RSYNC_EXCLUDE --rsync-path=\"$RSYNC_PATH\" -e \"$RSYNC_SSH_CMD\" \"$REMOTE_USER@$REMOTE_HOST:$1\" \"$local_file_storage_path\" > $RUN_DIR/obackup_rsync_output_$SCRIPT_PID 2>&1"
 	else
-		rsync_cmd="$(type -p $RSYNC_EXECUTABLE) $RSYNC_ARGS --stats --delete $RSYNC_EXCLUDE --rsync-path=\"$RSYNC_PATH\" \"$1\" \"$local_file_storage_path\" > $RUN_DIR/obackup_rsync_output_$SCRIPT_PID 2>&1"
+		rsync_cmd="$(type -p $RSYNC_EXECUTABLE) $RSYNC_ARGS --stats $RSYNC_DELETE $RSYNC_EXCLUDE --rsync-path=\"$RSYNC_PATH\" \"$1\" \"$local_file_storage_path\" > $RUN_DIR/obackup_rsync_output_$SCRIPT_PID 2>&1"
 	fi
 	#### Eval is used so the full command is processed without bash adding single quotes round variables
 	if [ $verbose -eq 1 ]
@@ -930,7 +963,7 @@ function FilesBackup
 	for BACKUP_TASK in $DIRECTORIES_SIMPLE_LIST
 	do
 		BACKUP_TASK=$(StripQuotes $BACKUP_TASK)
-		Log "Beginning file backup $BACKUP_TASK"
+		Log "Beginning recursive file backup on $BACKUP_TASK"
 		SECONDS_BEGIN=$SECONDS
 		Rsync $BACKUP_TASK &
 		child_pid=$!
@@ -954,10 +987,38 @@ function FilesBackup
 		CheckTotalExecutionTime
 	done
 
+        ## Also backup files at root of DIRECTORIES_RECURSE_LIST directories
+        for BACKUP_TASK in $DIRECTORIES_RECURSE_LIST
+        do
+                BACKUP_TASK="$(StripQuotes $BACKUP_TASK)"
+                Log "Beginning non recursive file backup on $BACKUP_TASK"
+                SECONDS_BEGIN=$SECONDS
+                Rsync $BACKUP_TASK "no-recurse" &
+		child_pid=$!
+                WaitForTaskCompletion $child_pid $SOFT_MAX_EXEC_TIME_FILE_TASK $HARD_MAX_EXEC_TIME_FILE_TASK
+                retval=$?
+                if [ $verbose -eq 1 ] && [ -f $RUN_DIR/obackup_rsync_output_$SCRIPT_PID ]
+                then
+                        Log "List:\n$(cat $RUN_DIR/obackup_rsync_output_$SCRIPT_PID)"
+                fi
+
+                if [ $retval -ne 0 ]
+                then
+                        LogError "Backup failed on remote files."
+                        if [ $verbose -eq 0 ] && [ -f $RUN_DIR/obackup_rsync_output_$SCRIPT_PID ]
+                        then
+                                LogError "$(cat $RUN_DIR/obackup_rsync_output_$SCRIPT_PID)"
+                        fi
+                else
+                        Log "Backup succeeded."
+                fi
+                CheckTotalExecutionTime
+        done
+
         for BACKUP_TASK in $DIRECTORIES_TO_BACKUP_RECURSE
         do
                 BACKUP_TASK=$(StripQuotes $BACKUP_TASK)
-                Log "Beginning file backup $BACKUP_TASK"
+                Log "Beginning recursive file backup on $BACKUP_TASK"
                 SECONDS_BEGIN=$SECONDS
                 Rsync $BACKUP_TASK "recurse" &
                 child_pid=$!
@@ -980,6 +1041,7 @@ function FilesBackup
                 fi
                 CheckTotalExecutionTime
         done
+		
 	IFS=$OLD_IFS
 }
 
@@ -1034,7 +1096,7 @@ function Init
 
 	if [ "$LOGFILE" == "" ]
 	then
-		if [ -d /var/log ]
+		if [ -w /var/log ]
 		then
 			LOG_FILE=/var/log/obackup_$OBACKUP_VERSION-$BACKUP_ID.log
 		else
@@ -1113,7 +1175,7 @@ function Init
                 RSYNC_ARGS=$RSYNC_ARGS"n"
                 DRY_WARNING="/!\ DRY RUN"
         fi
-        if [ "$BANDWIDTH" != "0" ]
+        if [ "$BANDWIDTH" != "" ] && [ "$BANDWIDTH" != "0" ]
         then
                 RSYNC_ARGS=$RSYNC_ARGS" --bwlimit=$BANDWIDTH"
         fi
@@ -1148,11 +1210,6 @@ function Main
 	if [ $dryrun -ne 1 ]
 	then
 		CreateLocalStorageDirectories
-	else
-		Log "DB backup list: $DATABASES_TO_BACKUP"
-	        Log "DB exclude list: $DATABASES_EXCLUDED_LIST"
-        	Log "Dirs backup list: $DIRECTORIES_TO_BACKUP"
-        	Log "Dirs exclude list: $DIRECTORIES_EXCLUDED_LIST"
 	fi
 
 	CheckSpaceRequirements
@@ -1249,6 +1306,7 @@ then
 		if [ $? == 0 ]
 		then
 			Init
+			GetOperatingSystem
 			DATE=$(date)
 			Log "--------------------------------------------------------------------"
 			Log "$DRY_WARNING $DATE - Obackup v$OBACKUP_VERSION script begin."