Initial v2.0 upload

CHANGELOG.md

@@ -1,9 +1,3 @@
-SHORT FUTURE IMPROVEMENTS
--------------------------
-
-- Rewrite rsync exclude patterns using \"pattern\" instead of escaped chars
-- Clean most of recursive task creation code
-
 KNOWN ISSUES
 ------------
 
@@ -11,16 +5,27 @@ KNOWN ISSUES
 - Bandwidth parameter is ignored for SQL backups
 - Missing symlink support when run from MSYS environment
 
-UNDER WORK
-----------
-
-- Commands like cp should have their stderr redirected to log file
-- Mysqldump must be checked for not telling success if a table is damaged (also check for event table error)
-- Mysqldump commands error msg must be logged
-
-
 CHANGELOG
 ---------
+
+README: FreeBSD execution needs mailer (not found), sudo missing, bash needed, sed missing (see if StripQuotes mandatory)
+
+! XX Dec 2015: obackup v2.0 released
+- Added reverse backup, now backups can be local, pushed or pulled to or from a remote system
+- Better fallback for SendAlert even if disk full
+- Added an alert email sent on warnings while backup script is running
+- Way better logging of errors in _GetDirectoriesSizeX, _BackupDatabaseX, _CreateStorageDirectoriesX
+- Added bogus config file checks & environment checks
+- Full code refactoring to use local and remote code once
+- Fully merged codebase with osync
+	- Added (much) more verbose debugging (and possibility to remove debug code to gain speed)
+	- Replace child_pid by $? directly, add a better sub process killer in TrapQuit
+	- Added some automatic checks in code, for _DEBUG mode (and _PARANOIA_DEBUG now)
+	- Improved Logging
+	- Updated obackup to be fully compliant with coding style
+- A long list of minor improvements
+
+v0-1.x - Jan 2013 - Oct 2015
 - New function to kill child processes
 - Fixed no_maxtime not honored
 - Improved some logging, also added highlighting to stdout errors

CODING_STYLE.TXT

@@ -0,0 +1,147 @@
+Coding style used for my bash projects (v2.1 Oct 2015)
+
+++++++ Header
+
+Always use the following header
+
+----BEGIN HEADER
+#!/usr/bin/env bash
+
+PROGRAM="program-name" # Long description
+AUTHOR="(L) 20XX-20YY by Orsiris \"Ozy\" de Jong"
+CONTACT="http://www.example.com me@example.com"
+PROGRAM_BUILD=YYYYMMDDVV
+
+## Optional instructions
+----END HEADER
+
+Using bind style versionning:
+YYYYMMDDVV (Year, Month, Day, Revision): Example: 2015012402 = 2nd revision of 24 Jan 2015
+
+#!/usr/bin/env bash instead of #!/bin/bash
+
+Change old scripts with
+for i in $(grep -r '#!/bin/bash' * |cut -f1 -d':'); do sed -i 's&#!/bin/bash&#!/usr/bin/env bash&g' $i; done
+
+
+type instead of type -p for bash test (other shells don't know -p)
+++++++ Indentation
+
+Using tabs
+Transform old shell scripts using unexpand command
+
+++++++ Comments
+
+Some command # comment
+## Some comment on a new line
+################################################# Some separation
+
+++++++ Work comments
+
+Whenever there is some idea to postpone, use #TODO[-version]:[dev-name:] some remark
+A marker must be left where on the line a dev is working (when the work isn't finished). Marker is #WIP:dev-name: some remark
+dev-name is mandatory if more than one person is coding
+Example: #TODO-v2.1:deajan: need to do something
+
+++++++ Variables
+
+All local variables are lowercase, separated by _ (ex: low_wait)
+All global variables full upercase, separated by _ (ex: EXEC_TIME)
+All environment variables (verbose, silent, debug, etc) have prefix _ and are full upercase, separated by _ (ex: _PARANOIA_DEBUG)
+
+++++++ Functions
+
+Every word in a function begins with an uppercase (ex: SomeFunctionDoesThings)
+
+Define functions this way. Use sed ':a;N;$!ba;s/\n{\n/ {\n/g' to adapt when opening bracket is on a new line.
+
+function something {
+	
+}
+
+If function has some arguments, use local variable names that are more readable than $1...$n. Explain via comments what those variables contain if needed.
+
+function anotherthing {
+	local var_name="${1}"
+	local other_var_name="${2}" # This variable contains stuff
+}
+
+Functions should always have return status
+function thirdthing {
+	some_command
+	return $?
+}
+
+++++++ Sub functions
+
+When a function is a subroutine of another function, it is called _SomethingAsSubFunction
+
+++++++ Function argument check
+
+Bash does not provide any checks against missing function arguments. Also, missing quotes can lead to an inconsistent number of arguments.
+Every function call will be checked by __CheckArguments which takes the number of arguments, $# (the real number of args given), the parent function name and the parent function's arguments.
+__CheckArguments will trigger a critical error if number of arguments if incorrect. This will also prevent silent typo errors.
+Ex:
+
+function Something {
+	local some="${1}"
+	local other="${2}"
+	local args="${3}"
+	__CheckArguments 3 $# $FUNCNAME "$*"
+
+__CheckArguments will only trigger if script is called with DEBUG=yes
+Also, with PARANOIA_DEBUG=yes, __CheckArguments will recount all arguments given by "$*" and compare. This can mislead if arguments contain spaces.
+
+++++++ If statements
+
+If statements will be fully written (word "if" must be used). then is written on the same line.
+(Use sed ':a;N;$!ba;s/]\n\t*then/]; then/g' to convert files to this format... Replace "],new line, zero or more tabs, then" by "; then")
+if [ something ]; then
+	stuff
+else
+	other stuff
+fi
+
+++++++ Logging
+
+A logging function is available with the following levels of logging:
+
+- DEBUG: Only log this when DEBUG flas is set in program. Any command forged for eval should be logged by this.
+- NOTICE: Standard messages
+- WARN: Requires attention
+- ERROR: Program produced an error but continues execution
+- CRITICAL: Program execution is halted
+
+++++++ Eval
+
+Most commands should be logged to a tmp file.
+The basic way of doing is:
+
+cmd='"something '$somevar'" > some_file 2>&1'
+eval $cmd &
+WaitForTaskCompletion $! 0 0 $FUNCNAME
+
+Remote commands should exist as:
+
+cmd=$SSH_CMD' "some; commands \"'$VARIABLE'\" some; other; commands" > some_file 2>&1'
+
+++++++ File variables
+
+All eval cmd should exit their content to a file called "$RUNDIR/osync.$FUNCNAME.$SCRIPT_PID"
+Dots are used instead of '_' so variables can be separated with a forbidden char in variables, so they get detected.
+
+++++++ Finding code errors
+
+Use shellcheck.net now and then (ignore SC2086 in our case)
+
+Use a low tech approach to find uneven number of quotes per line
+
+tr -cd "'\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}'
+tr -cd "\"\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}'
+
+++++++ ofunctions
+
+As obackup and osync share alot of common functions, ofunctions.sh will host all shared code.
+Dev programs n_osync.sh and n_obackup.sh will source ofunctions.sh
+Release programs will still include ofunctions.sh in order to enhance ease of use.
+

LICENCE.TXT

@@ -1,4 +1,4 @@
-Copyright (c) 2013, Orsiris "Ozy" de Jong. ozy@netpower.fr
+Copyright (c) 2013-2015, Orsiris "Ozy" de Jong. ozy@netpower.fr
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

dev/merge.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+
+## Merges ofunctions.sh and n_osync.sh into osync.sh
+
+PROGRAM=obackup
+FUNC_PATH=/home/git/common
+
+PARANOIA_DEBUG_LINE="__WITH_PARANOIA_DEBUG"
+PARANOIA_DEBUG_BEGIN="#__BEGIN_WITH_PARANOIA_DEBUG"
+PARANOIA_DEBUG_END="#__END_WITH_PARANOIA_DEBUG"
+
+function Unexpand {
+	unexpand n_$PROGRAM.sh > tmp_$PROGRAM.sh
+}
+
+function Merge {
+
+	sed "/source \"\/home\/git\/common\/ofunctions.sh\"/r /home/git/common/ofunctions.sh" tmp_$PROGRAM.sh | grep -v 'source "/home/git/common/ofunctions.sh"' > debug_$PROGRAM.sh
+	chmod +x debug_$PROGRAM.sh
+}
+
+function CleanDebug {
+
+# sed explanation
+#/pattern1/{         # if pattern1 is found
+#    p               # print it
+#    :a              # loop
+#        N           # and accumulate lines
+#    /pattern2/!ba   # until pattern2 is found
+#    s/.*\n//        # delete the part before pattern2
+#}
+#p
+
+	sed -n '/'$PARANOIA_DEBUG_BEGIN'/{p; :a; N; /'$PARANOIA_DEBUG_END'/!ba; s/.*\n//}; p' debug_$PROGRAM.sh | grep -v "$PARANOIA_DEBUG_LINE" > ../$PROGRAM.sh
+	chmod +x ../$PROGRAM.sh
+}
+
+function CopyCommons {
+	sed "s/\[prgname\]/$PROGRAM/g" /home/git/common/common_install.sh > ../install.sh
+	sed "s/\[prgname\]/$PROGRAM/g" /home/git/common/common_batch.sh > ../$PROGRAM-batch.sh
+	chmod +x ../install.sh
+	chmod +x ../obackup-batch.sh
+}
+
+Unexpand
+Merge
+CleanDebug
+rm -f tmp_$PROGRAM.sh
+CopyCommons

dev/ofunctions.sh

@@ -0,0 +1,885 @@
+FUNC_BUILD=2015111102
+## BEGIN Generic functions for osync & obackup written in 2013-2015 by Orsiris de Jong - http://www.netpower.fr - ozy@netpower.fr
+
+## type -p does not work on platforms other than linux (bash). If if does not work, always assume output is not a zero exitcode
+if ! type "$BASH" > /dev/null; then
+	echo "Please run this script only with bash shell. Tested on bash >= 3.2"
+	exit 127
+fi
+
+# Environment variables
+_DRYRUN=0
+_SILENT=0
+
+# Initial error status, logging 'WARN', 'ERROR' or 'CRITICAL' will enable alerts flags
+ERROR_ALERT=0
+WARN_ALERT=0
+
+## allow function call checks			#__WITH_PARANOIA_DEBUG
+if [ "$_PARANOIA_DEBUG" == "yes" ];then		#__WITH_PARANOIA_DEBUG
+	_DEBUG=yes				#__WITH_PARANOIA_DEBUG
+fi						#__WITH_PARANOIA_DEBUG
+
+## allow debugging from command line with _DEBUG=yes
+if [ ! "$_DEBUG" == "yes" ]; then
+	_DEBUG=no
+	SLEEP_TIME=.1
+	_VERBOSE=0
+else
+	SLEEP_TIME=1
+	trap 'TrapError ${LINENO} $?' ERR
+	_VERBOSE=1
+fi
+
+SCRIPT_PID=$$
+
+LOCAL_USER=$(whoami)
+LOCAL_HOST=$(hostname)
+
+## Default log file until config file is loaded
+if [ -w /var/log ]; then
+	LOG_FILE="/var/log/$PROGRAM.log"
+else
+	LOG_FILE="./$PROGRAM.log"
+fi
+
+## Default directory where to store temporary run files
+if [ -w /tmp ]; then
+	RUN_DIR=/tmp
+elif [ -w /var/tmp ]; then
+	RUN_DIR=/var/tmp
+else
+	RUN_DIR=.
+fi
+
+## Log a state message every $KEEP_LOGGING seconds. Should not be equal to soft or hard execution time so your log will not be unnecessary big.
+KEEP_LOGGING=1801
+
+## Correct output of sort command (language agnostic sorting)
+export LC_ALL=C
+
+# Standard alert mail body
+MAIL_ALERT_MSG="Execution of $PROGRAM instance $INSTANCE_ID on $(date) has warnings/errors."
+
+# Default alert attachment filename
+ALERT_LOG_FILE="$RUN_DIR/$PROGRAM.last.log"
+
+# Set error exit code if a piped command fails
+	set -o pipefail
+	set -o errtrace
+
+
+function Dummy {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+	sleep .1
+}
+
+function _Logger {
+	local svalue="${1}" # What to log to screen
+	local lvalue="${2:-$svalue}" # What to log to logfile, defaults to screen value
+	echo -e "$lvalue" >> "$LOG_FILE"
+
+	if [ $_SILENT -eq 0 ]; then
+		echo -e "$svalue"
+	fi
+}
+
+function Logger {
+	local value="${1}" # Sentence to log (in double quotes)
+	local level="${2}" # Log level: PARANOIA_DEBUG, DEBUG, NOTICE, WARN, ERROR, CRITIAL
+
+	# <OSYNC SPECIFIC> Special case in daemon mode we should timestamp instead of counting seconds
+	if [ "$sync_on_changes" == "1" ]; then
+		prefix="$(date) - "
+	else
+		prefix="TIME: $SECONDS - "
+	fi
+	# </OSYNC SPECIFIC>
+
+	if [ "$level" == "CRITICAL" ]; then
+		_Logger "$prefix\e[41m$value\e[0m" "$prefix$value"
+		ERROR_ALERT=1
+		return
+	elif [ "$level" == "ERROR" ]; then
+		_Logger "$prefix\e[91m$value\e[0m" "$prefix$value"
+		ERROR_ALERT=1
+		return
+	elif [ "$level" == "WARN" ]; then
+		_Logger "$prefix\e[93m$value\e[0m" "$prefix$value"
+		WARN_ALERT=1
+		return
+	elif [ "$level" == "NOTICE" ]; then
+		_Logger "$prefix$value"
+		return
+	elif [ "$level" == "DEBUG" ]; then
+		if [ "$_DEBUG" == "yes" ]; then
+			_Logger "$prefix$value"
+			return
+		fi
+	elif [ "$level" == "PARANOIA_DEBUG" ]; then		#__WITH_PARANOIA_DEBUG
+		if [ "$_PARANOIA_DEBUG" == "yes" ]; then	#__WITH_PARANOIA_DEBUG
+			_Logger "$prefix$value"			#__WITH_PARANOIA_DEBUG
+			return					#__WITH_PARANOIA_DEBUG
+		fi						#__WITH_PARANOIA_DEBUG
+	else
+		_Logger "\e[41mLogger function called without proper loglevel.\e[0m"
+		_Logger "$prefix$value"
+	fi
+}
+
+# Portable child (and grandchild) kill function tester under Linux, BSD and MacOS X
+function KillChilds {
+	local pid="${1}"
+	local self="${2:-false}"
+
+	if children="$(pgrep -P "$pid")"; then
+		for child in $children; do
+			KillChilds "$child" true
+		done
+	fi
+
+	# Try to kill nicely, if not, wait 30 seconds to let Trap actions happen before killing
+	if [ "$self" == true ]; then
+		kill -s SIGTERM "$pid" || (sleep 30 && kill -9 "$pid" &)
+	fi
+}
+
+function TrapError {
+	local job="$0"
+	local line="$1"
+	local code="${2:-1}"
+	if [ $_SILENT -eq 0 ]; then
+		echo -e " /!\ ERROR in ${job}: Near line ${line}, exit code ${code}"
+	fi
+}
+
+function Spinner {
+	if [ $_SILENT -eq 1 ]; then
+		return 0
+	fi
+
+	case $toggle
+	in
+	1)
+	echo -n " \ "
+	echo -ne "\r"
+	toggle="2"
+	;;
+
+	2)
+	echo -n " | "
+	echo -ne "\r"
+	toggle="3"
+	;;
+
+	3)
+	echo -n " / "
+	echo -ne "\r"
+	toggle="4"
+	;;
+
+	*)
+	echo -n " - "
+	echo -ne "\r"
+	toggle="1"
+	;;
+	esac
+}
+
+function SedStripQuotes {
+        echo $(echo $1 | sed "s/^\([\"']\)\(.*\)\1\$/\2/g")
+}
+
+function StripSingleQuotes {
+	local string="${1}"
+	string="${string/#\'/}" # Remove singlequote if it begins string
+	string="${string/%\'/}" # Remove singlequote if it ends string
+	echo "$string"
+}
+
+function StripDoubleQuotes {
+	local string="${1}"
+	string="${string/#\"/}"
+	string="${string/%\"/}"
+	echo "$string"
+}
+
+function StripQuotes {
+	local string="${1}"
+	echo "$(StripSingleQuotes $(StripDoubleQuotes $string))"
+}
+
+function EscapeSpaces {
+	local string="${1}" # String on which spaces will be escaped
+	echo "${string// /\ }"
+}
+
+function IsNumeric {
+	eval "local value=\"${1}\"" # Needed so variable variables can be processed
+
+	local re="^-?[0-9]+([.][0-9]+)?$"
+	if [[ $value =~ $re ]]; then
+		echo 1
+	else
+		echo 0
+	fi
+}
+
+function CleanUp {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ "$_DEBUG" != "yes" ]; then
+		rm -f "$RUN_DIR/$PROGRAM."*".$SCRIPT_PID"
+	fi
+}
+
+function SendAlert {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ "$_DEBUG" == "yes" ]; then
+		Logger "Debug mode, no warning email will be sent." "NOTICE"
+		return 0
+	fi
+
+	# <OSYNC SPECIFIC>
+	if [ "$_QUICK_SYNC" == "2" ]; then
+		Logger "Current task is a quicksync task. Will not send any alert." "NOTICE"
+		return 0
+	fi
+	# </OSYNC SPECIFIC>
+
+	eval "cat \"$LOG_FILE\" $COMPRESSION_PROGRAM > $ALERT_LOG_FILE"
+	MAIL_ALERT_MSG="$MAIL_ALERT_MSG"$'\n\n'$(tail -n 25 "$LOG_FILE")
+	if [ $ERROR_ALERT -eq 1 ]; then
+		subject="Error alert for $INSTANCE_ID"
+	elif [ $WARN_ALERT -eq 1 ]; then
+		subject="Warning alert for $INSTANCE_ID"
+	else
+		subject="Alert for $INSTANCE_ID"
+	fi
+
+	# Need better fallback if mail sending does not succeed
+	if type mutt > /dev/null 2>&1 ; then
+		echo "$MAIL_ALERT_MSG" | $(type -p mutt) -x -s "$subject" $DESTINATION_MAILS -a "$ALERT_LOG_FILE"
+		if [ $? != 0 ]; then
+			Logger "WARNING: Cannot send alert email via $(type -p mutt) !!!" "WARN"
+		else
+			Logger "Sent alert mail using mutt." "NOTICE"
+			return 0
+		fi
+	fi
+
+	if type mail > /dev/null 2>&1 ; then
+		echo "$MAIL_ALERT_MSG" | $(type -p mail) -a "$ALERT_LOG_FILE" -s "$subject" $DESTINATION_MAILS
+		if [ $? != 0 ]; then
+			Logger "WARNING: Cannot send alert email via $(type -p mail) with attachments !!!" "WARN"
+			echo "$MAIL_ALERT_MSG" | $(type -p mail) -s "$subject" $DESTINATION_MAILS
+			if [ $? != 0 ]; then
+				Logger "WARNING: Cannot send alert email via $(type -p mail) without attachments !!!" "WARN"
+			else
+				Logger "Sent alert mail using mail command without attachment." "NOTICE"
+				return 0
+			fi
+		else
+			Logger "Sent alert mail using mail command." "NOTICE"
+			return 0
+		fi
+	fi
+
+	if type sendmail > /dev/null 2>&1 ; then
+		echo -e "$subject\r\n$MAIL_ALERT_MSG" | $(type -p sendmail) $DESTINATION_MAILS
+		if [ $? != 0 ]; then
+			Logger "WARNING: Cannot send alert email via $(type -p sendmail) !!!" "WARN"
+		else
+			Logger "Sent alert mail using sendmail command without attachment." "NOTICE"
+			return 0
+		fi
+	fi
+
+	if type sendemail > /dev/null 2>&1 ; then
+		if [ "$SMTP_USER" != "" ] && [ "$SMTP_PASSWORD" != "" ]; then
+			SMTP_OPTIONS="-xu $SMTP_USER -xp $SMTP_PASSWORD"
+		else
+			SMTP_OPTIONS=""
+		fi
+		$(type -p sendemail) -f $SENDER_MAIL -t $DESTINATION_MAILS -u "$subject" -m "$MAIL_ALERT_MSG" -s $SMTP_SERVER $SMTP_OPTIONS > /dev/null 2>&1
+		if [ $? != 0 ]; then
+			Logger "WARNING: Cannot send alert email via $(type -p sendemail) !!!" "WARN"
+		else
+			Logger "Sent alert mail using sendemail command without attachment." "NOTICE"
+			return 0
+		fi
+	fi
+
+	# If function has not returned 0 yet, assume it's critical that no alert can be sent
+	Logger "/!\ CRITICAL: Cannot send alert" "ERROR" # Is not marked critical because execution must continue
+
+	# Delete tmp log file
+	if [ -f "$ALERT_LOG_FILE" ]; then
+		rm "$ALERT_LOG_FILE"
+	fi
+}
+
+function LoadConfigFile {
+	__CheckArguments 1 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	local config_file="${1}"
+
+	if [ ! -f "$config_file" ]; then
+		Logger "Cannot load configuration file [$config_file]. Cannot start." "CRITICAL"
+		exit 1
+	elif [[ "$1" != *".conf" ]]; then
+		Logger "Wrong configuration file supplied [$config_file]. Cannot start." "CRITICAL"
+		exit 1
+	else
+		grep '^[^ ]*=[^;&]*' "$config_file" > "$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID" # WITHOUT COMMENTS
+		# Shellcheck source=./sync.conf
+		source "$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID"
+	fi
+}
+
+function GetLocalOS {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	local local_os_var=$(uname -spio 2>&1)
+	if [ $? != 0 ]; then
+		local local_os_var=$(uname -v 2>&1)
+		if [ $? != 0 ]; then
+			local local_os_var=($uname)
+		fi
+	fi
+
+	case $local_os_var in
+		*"Linux"*)
+		LOCAL_OS="Linux"
+		;;
+		*"BSD"*)
+		LOCAL_OS="BSD"
+		;;
+		*"MINGW32"*)
+		LOCAL_OS="msys"
+		;;
+		*"Darwin"*)
+		LOCAL_OS="MacOSX"
+		;;
+		*)
+		Logger "Running on >> $local_os_var << not supported. Please report to the author." "ERROR"
+		exit 1
+		;;
+	esac
+	Logger "Local OS: [$local_os_var]." "DEBUG"
+}
+
+function GetRemoteOS {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ "$REMOTE_OPERATION" == "yes" ]; then
+		CheckConnectivity3rdPartyHosts
+		CheckConnectivityRemoteHost
+		local cmd=$SSH_CMD' "uname -spio" > "'$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID'" 2>&1'
+		Logger "cmd: $cmd" "DEBUG"
+		eval "$cmd" &
+		WaitForTaskCompletion $! 120 240 $FUNCNAME"-1"
+		retval=$?
+		if [ $retval != 0 ]; then
+			local cmd=$SSH_CMD' "uname -v" > "'$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID'" 2>&1'
+			Logger "cmd: $cmd" "DEBUG"
+			eval "$cmd" &
+			WaitForTaskCompletion $! 120 240 $FUNCNAME"-2"
+			retval=$?
+			if [ $retval != 0 ]; then
+				local cmd=$SSH_CMD' "uname" > "'$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID'" 2>&1'
+				Logger "cmd: $cmd" "DEBUG"
+				eval "$cmd" &
+				WaitForTaskCompletion $! 120 240 $FUNCNAME"-3"
+				retval=$?
+				if [ $retval != 0 ]; then
+					Logger "Cannot Get remote OS type." "ERROR"
+				fi
+			fi
+		fi
+
+		local remote_os_var=$(cat $RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID)
+
+		case $remote_os_var in
+			*"Linux"*)
+			REMOTE_OS="Linux"
+			;;
+			*"BSD"*)
+			REMOTE_OS="BSD"
+			;;
+			*"MINGW32"*)
+			REMOTE_OS="msys"
+			;;
+			*"Darwin"*)
+			REMOTE_OS="MacOSX"
+			;;
+			*"ssh"*|*"SSH"*)
+			Logger "Cannot connect to remote system." "CRITICAL"
+			exit 1
+			;;
+			*)
+			Logger "Running on remote OS failed. Please report to the author if the OS is not supported." "CRITICAL"
+			Logger "Remote OS said:\n$remote_os_var" "CRITICAL"
+			exit 1
+		esac
+
+		Logger "Remote OS: [$remote_os_var]." "DEBUG"
+	fi
+}
+
+function WaitForTaskCompletion {
+	local pid="${1}" # pid to wait for
+	local soft_max_time="${2}" # If program with pid $pid takes longer than $soft_max_time seconds, will log a warning, unless $soft_max_time equals 0.
+	local hard_max_time="${3}" # If program with pid $pid takes longer than $hard_max_time seconds, will stop execution, unless $hard_max_time equals 0.
+	local caller_name="${4}" # Who called this function
+	Logger "$FUNCNAME called by [$caller_name]." "PARANOIA_DEBUG"	#__WITH_PARANOIA_DEBUG
+	__CheckArguments 4 $# $FUNCNAME "$@"				#__WITH_PARANOIA_DEBUG
+
+	local soft_alert=0 # Does a soft alert need to be triggered, if yes, send an alert once
+	local log_ttime=0 # local time instance for comparaison
+
+	local seconds_begin=$SECONDS # Seconds since the beginning of the script
+	local exec_time=0 # Seconds since the beginning of this function
+
+	while eval "$PROCESS_TEST_CMD" > /dev/null
+	do
+		Spinner
+		exec_time=$(($SECONDS - $seconds_begin))
+		if [ $((($exec_time + 1) % $KEEP_LOGGING)) -eq 0 ]; then
+			if [ $log_ttime -ne $exec_time ]; then
+				log_ttime=$exec_time
+				Logger "Current task still running." "NOTICE"
+			fi
+		fi
+		if [ $exec_time -gt $soft_max_time ]; then
+			if [ $soft_alert -eq 0 ] && [ $soft_max_time -ne 0 ]; then
+				Logger "Max soft execution time exceeded for task." "WARN"
+				soft_alert=1
+				SendAlert
+
+			fi
+			if [ $exec_time -gt $hard_max_time ] && [ $hard_max_time -ne 0 ]; then
+				Logger "Max hard execution time exceeded for task. Stopping task execution." "ERROR"
+				kill -s SIGTERM $pid
+				if [ $? == 0 ]; then
+					Logger "Task stopped succesfully" "NOTICE"
+				else
+					Logger "Sending SIGTERM to proces failed. Trying the hard way." "ERROR"
+					sleep 5 && kill -9 $pid
+					if [ $? != 0 ]; then
+						Logger "Could not stop task." "ERROR"
+					fi
+				fi
+				return 1
+			fi
+		fi
+		sleep $SLEEP_TIME
+	done
+	wait $pid
+	local retval=$?
+	Logger "$FUNCNAME ended for [$caller_name] with status $retval." "PARANOIA_DEBUG"	#__WITH_PARANOIA_DEBUG
+	return $retval
+}
+
+function WaitForCompletion {
+	local pid="${1}" # pid to wait for
+	local soft_max_time="${2}" # If program with pid $pid takes longer than $soft_max_time seconds, will log a warning, unless $soft_max_time equals 0.
+	local hard_max_time="${3}" # If program with pid $pid takes longer than $hard_max_time seconds, will stop execution, unless $hard_max_time equals 0.
+	local caller_name="${4}" # Who called this function
+	Logger "$FUNCNAME called by [$caller_name]" "PARANOIA_DEBUG"	#__WITH_PARANOIA_DEBUG
+	__CheckArguments 4 $# $FUNCNAME "$@"				#__WITH_PARANOIA_DEBUG
+
+	local soft_alert=0 # Does a soft alert need to be triggered, if yes, send an alert once
+	local log_ttime=0 # local time instance for comparaison
+
+	local seconds_begin=$SECONDS # Seconds since the beginning of the script
+	local exec_time=0 # Seconds since the beginning of this function
+
+	while eval "$PROCESS_TEST_CMD" > /dev/null
+	do
+		Spinner
+		if [ $((($SECONDS + 1) % $KEEP_LOGGING)) -eq 0 ]; then
+			if [ $log_time -ne $SECONDS ]; then
+				log_time=$SECONDS
+				Logger "Current task still running." "NOTICE"
+			fi
+		fi
+		if [ $SECONDS -gt $soft_max_time ]; then
+			if [ $soft_alert -eq 0 ] && [ $soft_max_time != 0 ]; then
+				Logger "Max soft execution time exceeded for script." "WARN"
+				soft_alert=1
+				SendAlert
+			fi
+			if [ $SECONDS -gt $hard_max_time ] && [ $hard_max_time != 0 ]; then
+				Logger "Max hard execution time exceeded for script. Stopping current task execution." "ERROR"
+				kill -s SIGTERM $pid
+				if [ $? == 0 ]; then
+					Logger "Task stopped succesfully" "NOTICE"
+				else
+					Logger "Sending SIGTERM to proces failed. Trying the hard way." "ERROR"
+					kill -9 $pid
+					if [ $? != 0 ]; then
+						Logger "Could not stop task." "ERROR"
+					fi
+				fi
+				return 1
+			fi
+		fi
+		sleep $SLEEP_TIME
+	done
+	wait $pid
+	retval=$?
+	Logger "$FUNCNAME ended for [$caller_name] with status $retval." "PARANOIA_DEBUG"	#__WITH_PARANOIA_DEBUG
+	return $retval
+}
+
+function RunLocalCommand {
+	local command="${1}" # Command to run
+	local hard_max_time="${2}" # Max time to wait for command to compleet
+	__CheckArguments 2 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ $_DRYRUN -ne 0 ]; then
+		Logger "Dryrun: Local command [$command] not run." "NOTICE"
+		return 1
+	fi
+	Logger "Running command [$command] on local host." "NOTICE"
+	eval "$command" > "$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID" 2>&1 &
+	WaitForTaskCompletion $! 0 $hard_max_time $FUNCNAME
+	retval=$?
+	if [ $retval -eq 0 ]; then
+		Logger "Command succeded." "NOTICE"
+	else
+		Logger "Command failed." "ERROR"
+	fi
+
+	if [ $_VERBOSE -eq 1 ] || [ $retval -ne 0 ]; then
+		Logger "Command output:\n$(cat $RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID)" "NOTICE"
+	fi
+
+	if [ "$STOP_ON_CMD_ERROR" == "yes" ] && [ $retval -ne 0 ]; then
+		Logger "Stopping on command execution error." "CRITICAL"
+		exit 1
+	fi
+}
+
+## Runs remote command $1 and waits for completition in $2 seconds
+function RunRemoteCommand {
+	local command="${1}" # Command to run
+	local hard_max_time="${2}" # Max time to wait for command to compleet
+	__CheckArguments 2 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	CheckConnectivity3rdPartyHosts
+	CheckConnectivityRemoteHost
+	if [ $_DRYRUN -ne 0 ]; then
+		Logger "Dryrun: Local command [$command] not run." "NOTICE"
+		return 1
+	fi
+	Logger "Running command [$command] on remote host." "NOTICE"
+	cmd=$SSH_CMD' "$command" > "'$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID'" 2>&1'
+	Logger "cmd: $cmd" "DEBUG"
+	eval "$cmd" &
+	WaitForTaskCompletion $! 0 $hard_max_time $FUNCNAME
+	retval=$?
+	if [ $retval -eq 0 ]; then
+		Logger "Command succeded." "NOTICE"
+	else
+		Logger "Command failed." "ERROR"
+	fi
+
+	if [ -f "$RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID" ] && ([ $_VERBOSE -eq 1 ] || [ $retval -ne 0 ])
+	then
+		Logger "Command output:\n$(cat $RUN_DIR/$PROGRAM.$FUNCNAME.$SCRIPT_PID)" "NOTICE"
+	fi
+
+	if [ "$STOP_ON_CMD_ERROR" == "yes" ] && [ $retval -ne 0 ]; then
+		Logger "Stopping on command execution error." "CRITICAL"
+		exit 1
+	fi
+}
+
+function RunBeforeHook {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ "$LOCAL_RUN_BEFORE_CMD" != "" ]; then
+		RunLocalCommand "$LOCAL_RUN_BEFORE_CMD" $MAX_EXEC_TIME_PER_CMD_BEFORE
+	fi
+
+	if [ "$REMOTE_RUN_BEFORE_CMD" != "" ]; then
+		RunRemoteCommand "$REMOTE_RUN_BEFORE_CMD" $MAX_EXEC_TIME_PER_CMD_BEFORE
+	fi
+}
+
+function RunAfterHook {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ "$LOCAL_RUN_AFTER_CMD" != "" ]; then
+		RunLocalCommand "$LOCAL_RUN_AFTER_CMD" $MAX_EXEC_TIME_PER_CMD_AFTER
+	fi
+
+	if [ "$REMOTE_RUN_AFTER_CMD" != "" ]; then
+		RunRemoteCommand "$REMOTE_RUN_AFTER_CMD" $MAX_EXEC_TIME_PER_CMD_AFTER
+	fi
+}
+
+function CheckConnectivityRemoteHost {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ "$_PARANOIA_DEBUG" != "yes" ]; then # Do not loose time in paranoia debug
+
+		if [ "$REMOTE_HOST_PING" != "no" ] && [ "$REMOTE_OPERATION" != "no" ]; then
+			eval "$PING_CMD $REMOTE_HOST > /dev/null 2>&1" &
+			WaitForTaskCompletion $! 180 180 $FUNCNAME
+			if [ $? != 0 ]; then
+				Logger "Cannot ping $REMOTE_HOST" "CRITICAL"
+				return 1
+			fi
+		fi
+	fi
+}
+
+function CheckConnectivity3rdPartyHosts {
+	__CheckArguments 0 $# $FUNCNAME "$@"	#__WITH_PARANOIA_DEBUG
+
+	if [ "$_PARANOIA_DEBUG" != "yes" ]; then # Do not loose time in paranoia debug
+
+		if [ "$REMOTE_3RD_PARTY_HOSTS" != "" ]; then
+			remote_3rd_party_success=0
+			OLD_IFS=$IFS
+			IFS=$' \t\n'
+			for i in $REMOTE_3RD_PARTY_HOSTS
+			do
+				eval "$PING_CMD $i > /dev/null 2>&1" &
+				WaitForTaskCompletion $! 360 360 $FUNCNAME
+				if [ $? != 0 ]; then
+					Logger "Cannot ping 3rd party host $i" "WARN"
+				else
+					remote_3rd_party_success=1
+				fi
+			done
+			IFS=$OLD_IFS
+			if [ $remote_3rd_party_success -ne 1 ]; then
+				Logger "No remote 3rd party host responded to ping. No internet ?" "CRITICAL"
+				return 1
+			fi
+		fi
+	fi
+}
+
+#__BEGIN_WITH_PARANOIA_DEBUG
+function __CheckArguments {
+	# Checks the number of arguments of a function and raises an error if some are missing
+
+	if [ "$_DEBUG" == "yes" ]; then
+                local number_of_arguments="${1}" # Number of arguments the tested function should have
+                local number_of_given_arguments="${2}" # Number of arguments that have been passed
+                local function_name="${3}" # Function name that called __CheckArguments
+
+		if [ "$_PARANOIA_DEBUG" == "yes" ]; then
+			Logger "Entering function [$function_name]." "DEBUG"
+		fi
+
+                # All arguments of the function to check are passed as array in ${4} (the function call waits for $@)
+                # If any of the arguments contains spaces, bash things there are two aguments
+                # In order to avoid this, we need to iterate over ${4} and count
+
+                local iterate=4
+                local fetch_arguments=1
+                local arg_list=""
+                while [ $fetch_arguments -eq 1 ]; do
+                        cmd='argument=${'$iterate'}'
+                        eval $cmd
+                        if [ "$argument" = "" ]; then
+                                fetch_arguments=0
+                        else
+                                arg_list="$arg_list [Argument $(($iterate-3)): $argument]"
+                                iterate=$(($iterate+1))
+                        fi
+                done
+                local counted_arguments=$((iterate-4))
+
+                if [ $counted_arguments -ne $number_of_arguments ]; then
+                        Logger "Function $function_name may have inconsistent number of arguments. Expected: $number_of_arguments, count: $counted_arguments, see log file." "ERROR"
+                        Logger "Arguments passed: $arg_list" "ERROR"
+                fi
+	fi
+}
+
+
+function old__CheckArguments {
+	# Checks the number of arguments and raises an error if some are missing
+	if [ "$_DEBUG" == "yes" ]; then
+
+		local number_of_arguments="${1}" # Number of arguments a function should have
+		local number_of_given_arguments="${2}" # Number of arguments that have been passed
+		local function_name="${3}" # Function name that called __CheckArguments
+		local arguments="${4}" # All other arguments
+
+		if [ "$_PARANOIA_DEBUG" == "yes" ]; then
+			Logger "Entering function [$function_name]." "DEBUG"
+
+			# Paranoia check... Can help finding empty arguments. __CheckArguments should be grepped out in production builds.
+			local count=-3 # Number of arguments minus the function calls for __CheckArguments
+			for i in $@; do
+				count=$((count + 1))
+			done
+			if [ $count -ne $1 ]; then
+				Logger "Function $function_name may have inconsistent number of arguments. Expected: $number_of_arguments, count: $count, see log file." "WARN"
+				echo "Argument list (including checks): $*" >> "$LOG_FILE"
+			fi
+		fi
+
+		if [ $number_of_arguments -ne $number_of_given_arguments ]; then
+			Logger "Inconsistnent number of arguments in $function_name. Should have $number_of_arguments arguments, has $number_of_given_arguments arguments, see log file." "CRITICAL"
+			# Cannot user Logger here because $@ is a list of arguments
+			echo "Argumnt list: $4" >> "$LOG_FILE"
+		fi
+
+	fi
+}
+#__END_WITH_PARANOIA_DEBUG
+
+function PreInit {
+	 __CheckArguments 0 $# $FUNCNAME "$@"    #__WITH_PARANOIA_DEBUG
+
+	## SSH compression
+        if [ "$SSH_COMPRESSION" != "no" ]; then
+                SSH_COMP=-C
+        else
+                SSH_COMP=
+        fi
+
+        ## Support for older config files without RSYNC_EXECUTABLE option
+        if [ "$RSYNC_EXECUTABLE" == "" ]; then
+                RSYNC_EXECUTABLE=rsync
+        fi
+
+        ## Sudo execution option
+        if [ "$SUDO_EXEC" == "yes" ]; then
+                if [ "$RSYNC_REMOTE_PATH" != "" ]; then
+                        RSYNC_PATH="sudo $RSYNC_REMOTE_PATH/$RSYNC_EXECUTABLE"
+                else
+                        RSYNC_PATH="sudo $RSYNC_EXECUTABLE"
+                fi
+                COMMAND_SUDO="sudo"
+        else
+                if [ "$RSYNC_REMOTE_PATH" != "" ]; then
+                        RSYNC_PATH="$RSYNC_REMOTE_PATH/$RSYNC_EXECUTABLE"
+                else
+                        RSYNC_PATH="$RSYNC_EXECUTABLE"
+                fi
+                COMMAND_SUDO=""
+        fi
+
+	 ## Set rsync default arguments
+        RSYNC_ARGS="-rlptgoD"
+
+        if [ "$PRESERVE_ACL" == "yes" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -A"
+        fi
+        if [ "$PRESERVE_XATTR" == "yes" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -X"
+        fi
+        if [ "$RSYNC_COMPRESS" == "yes" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -z"
+        fi
+        if [ "$COPY_SYMLINKS" == "yes" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -L"
+        fi
+        if [ "$KEEP_DIRLINKS" == "yes" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -K"
+        fi
+        if [ "$PRESERVE_HARDLINKS" == "yes" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -H"
+        fi
+        if [ "$CHECKSUM" == "yes" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" --checksum"
+        fi
+	if [ $_DRYRUN -eq 1 ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -n"
+                DRY_WARNING="/!\ DRY RUN"
+        fi
+        if [ "$BANDWIDTH" != "" ] && [ "$BANDWIDTH" != "0" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" --bwlimit=$BANDWIDTH"
+        fi
+
+	 ## Set compression executable and extension
+        COMPRESSION_LEVEL=3
+        if type xz > /dev/null 2>&1
+        then
+                COMPRESSION_PROGRAM="| xz -$COMPRESSION_LEVEL"
+                COMPRESSION_EXTENSION=.xz
+        elif type lzma > /dev/null 2>&1
+        then
+                COMPRESSION_PROGRAM="| lzma -$COMPRESSION_LEVEL"
+                COMPRESSION_EXTENSION=.lzma
+        elif type pigz > /dev/null 2>&1
+        then
+                COMPRESSION_PROGRAM="| pigz -$COMPRESSION_LEVEL"
+                COMPRESSION_EXTENSION=.gz
+                COMPRESSION_OPTIONS=--rsyncable
+        elif type gzip > /dev/null 2>&1
+        then
+                COMPRESSION_PROGRAM="| gzip -$COMPRESSION_LEVEL"
+                COMPRESSION_EXTENSION=.gz
+                COMPRESSION_OPTIONS=--rsyncable
+        else
+                COMPRESSION_PROGRAM=
+                COMPRESSION_EXTENSION=
+        fi
+        ALERT_LOG_FILE="$ALERT_LOG_FILE$COMPRESSION_EXTENSION"
+}
+
+function PostInit {
+        __CheckArguments 0 $# $FUNCNAME "$@"    #__WITH_PARANOIA_DEBUG
+
+	# Define remote commands
+        SSH_CMD="$(type -p ssh) $SSH_COMP -i $SSH_RSA_PRIVATE_KEY $REMOTE_USER@$REMOTE_HOST -p $REMOTE_PORT"
+        SCP_CMD="$(type -p scp) $SSH_COMP -i $SSH_RSA_PRIVATE_KEY -P $REMOTE_PORT"
+        RSYNC_SSH_CMD="$(type -p ssh) $SSH_COMP -i $SSH_RSA_PRIVATE_KEY -p $REMOTE_PORT"
+}
+
+function InitLocalOSSettings {
+        __CheckArguments 0 $# $FUNCNAME "$@"    #__WITH_PARANOIA_DEBUG
+
+        ## If running under Msys, some commands do not run the same way
+        ## Using mingw version of find instead of windows one
+        ## Getting running processes is quite different
+        ## Ping command is not the same
+        if [ "$LOCAL_OS" == "msys" ]; then
+                FIND_CMD=$(dirname $BASH)/find
+                #TODO: The following command needs to be checked on msys. Does the $1 variable substitution work ?
+                # PROCESS_TEST_CMD assumes there is a variable $pid
+                PROCESS_TEST_CMD='ps -a | awk "{\$1=\$1}\$1" | awk "{print \$1}" | grep $pid'
+                PING_CMD="ping -n 2"
+        else
+                FIND_CMD=find
+                # PROCESS_TEST_CMD assumes there is a variable $pid
+                PROCESS_TEST_CMD='ps -p$pid'
+                PING_CMD="ping -c 2 -i .2"
+        fi
+
+        ## Stat command has different syntax on Linux and FreeBSD/MacOSX
+        if [ "$LOCAL_OS" == "MacOSX" ] || [ "$LOCAL_OS" == "BSD" ]; then
+                STAT_CMD="stat -f \"%Sm\""
+        else
+                STAT_CMD="stat --format %y"
+        fi
+}
+
+function InitRemoteOSSettings {
+        __CheckArguments 0 $# $FUNCNAME "$@"    #__WITH_PARANOIA_DEBUG
+
+        ## MacOSX does not use the -E parameter like Linux or BSD does (-E is mapped to extended attrs instead of preserve executability)
+        if [ "$LOCAL_OS" != "MacOSX" ] && [ "$REMOTE_OS" != "MacOSX" ]; then
+                RSYNC_ARGS=$RSYNC_ARGS" -E"
+        fi
+
+        if [ "$REMOTE_OS" == "msys" ]; then
+                REMOTE_FIND_CMD=$(dirname $BASH)/find
+        else
+                REMOTE_FIND_CMD=find
+        fi
+}
+
+## END Generic functions

exclude.list.example

@@ -1,7 +0,0 @@
-.AppleDouble/
-._*
-.DS_Store
-Thumbs.db
-System Volume Information
-$Recycle.Bin
-

exclude.list.example

@@ -0,0 +1 @@
+/home/git/common/exclude.list.example

host_backup.conf

@@ -1,83 +1,86 @@
-#!/bin/bash
+###### Local / Remote backup script for files & databases
+###### (L) 2013-2015 by Ozy de Jong (www.netpower.fr)
+###### obackup v2.x config file rev 2015111102
 
-###### Remote (or local) backup script for files & databases
+###### GENERAL BACKUP OPTIONS
-###### (L) 2013 by Ozy de Jong (www.netpower.fr)
-###### Config file rev 2015090801
 
-## ---------- GENERAL BACKUP OPTIONS
+## Backup identification string.
+INSTANCE_ID="test-backup"
 
-## Backup identification name.
+## Log file location. Leaving this empty will create log file at /var/log/obackup.INSTANCE_ID.log (or current directory if /var/log doesn't exist).
-BACKUP_ID="your backup name"
-
-## Log file location. Leaving this empty will create log file at /var/log/obackup_version_BACKUP_ID.log (or current directory if /var/log doesn't exist)
 LOGFILE=""
 
-## Backup databases
+## Elements to backup
-BACKUP_SQL=no
+SQL_BACKUP=yes
-## Backup files
+FILE_BACKUP=yes
-BACKUP_FILES=yes
 
-## ---------- LOCAL BACKUP STORAGE OPTIONS
+## Backups can be done local, pulled from another server or pushed to a backup server. Available options are [local,pull,push].
+## Pulled backups are the safest option, as the backup server contains the RSA key and cannot be compromised by another server.
+BACKUP_TYPE=local
 
-## Local storage paths where to put backups
+###### BACKUP STORAGE
-LOCAL_SQL_STORAGE="/home/storage/backup/sql"
+
-LOCAL_FILE_STORAGE="/home/storage/backup/files"
+## Storage paths of the backups (absolute paths of the local or remote system)
+SQL_STORAGE="/home/storage/backup/sql"
+FILE_STORAGE="/home/storage/backup/files"
+
+## Backup encryption using GPG and duplicity. Feature not ready yet.
+ENCRYPTION=no
 
 ## Create backup directories if they do not exist
 CREATE_DIRS=yes
 
 ## Keep absolute source path in your backup, eg: /your/backup/storage/the/remote/server/files
-## You should leave this enabled if you intend to use 'backup task division' functionality of OBackup, or everything will end up in the same directory.
+## You should leave this enabled if you intend to use 'backup task division' functionality of oBackup, or everything will end up in the same directory.
-LOCAL_STORAGE_KEEP_ABSOLUTE_PATHS=yes
+KEEP_ABSOLUTE_PATHS=yes
 
-## Generate an alert if backup size is lower than given value in Kb (this can also help identifying empty mount dirs)
+## Generate an alert if backup size is lower than given value in Kb (this can also help identifying empty mount dirs).
 BACKUP_SIZE_MINIMUM=1024
-## You may disable testing backup size
-DISABLE_GET_BACKUP_FILE_SIZE=no
 
-## Generate an alert if local storage free space is lower than given value in Kb. Keep in mind that disabling backup file size test will only test min space against SQL backup size.
+## Check backup size before proceeding
-LOCAL_STORAGE_WARN_MIN_SPACE=1048576
+GET_BACKUP_SIZE=yes
 
-## ---------- MISC OPTIONS
+## Generate an alert if storage free space is lower than given value in Kb.
+## Keep in mind that disabling backup file size test will only test min space against SQL backup size.
+SQL_WARN_MIN_SPACE=1048576
+FILE_WARN_MIN_SPACE=1048576
 
-## Bandwidth limit Kbytes / second for file backups. Leave 0 to disable limitation.
+## If enabled, backups will be processed as superuser. See documentation for /etc/sudoers configuration ("find", "du", "tee" and "rsync" need to be allowed). Requiretty needs to be disabled.
-BANDWIDTH=0
+SUDO_EXEC=yes
 
-## If enabled, file backups will be processed as superuser. See documentation for /etc/sudoers configuration ("find", "du" and "rsync" need to be allowed). Requiretty needs to be disabled.
+###### REMOTE ONLY OPTIONS
-SUDO_EXEC=no
-## Paranoia option. Don't change this unless you read the documentation.
-RSYNC_EXECUTABLE=rsync
 
-## ---------- REMOTE BACKUP OPTIONS
+## In case of pulled or pushed backups, remote system URI needs to be supplied.
+#REMOTE_SYSTEM_URI="ssh://backupuser@remote.system.tld:22/"
+REMOTE_SYSTEM_URI="ssh://backupmaster@mailer.mvacances.org:49998/"
+
+## You can specify a RSA key (please use full path). If not defined, the default ~/.ssh/id_rsa will be used. See documentation for further information.
+SSH_RSA_PRIVATE_KEY="/root/.ssh/id_rsa.srvmailing"
 
-## The following options allow this Obackup instance to connect to a remote system via an ssh tunnel.
-## Needs public RSA key need to be put into ~/.ssh/authorized_keys in remote users home directory.
-REMOTE_BACKUP=no
-SSH_RSA_PRIVATE_KEY=~/.ssh/id_rsa
-REMOTE_USER=backupuser
-REMOTE_HOST=yourhost.local
-REMOTE_PORT=22
 ## ssh compression should be used unless your remote connection is good enough (LAN)
 SSH_COMPRESSION=yes
+
 ## Remote rsync executable path. Leave this empty in most cases
 RSYNC_REMOTE_PATH=""
 
 ## Check for connectivity to remote host before launching remote backup tasks. Be sure the hosts responds to ping. Failing to ping will skip current task.
 REMOTE_HOST_PING=yes
+
 ## Check for internet access by pinging one or more 3rd party hosts before remote backup tasks. Leave empty if you don't want this check to be be performed. Failing to ping will skip current task.
 REMOTE_3RD_PARTY_HOSTS="www.kernel.org www.google.com"
 
-## ---------- DATABASE BACKUP OPTIONS
+###### DATABASE SPECIFIC OPTIONS
 
 ## Database backup user
-SQL_USER=backupuser
+SQL_USER=backupmaster
+SQL_USER=root #DEBUG
 
 ## Enabling the following option will save all databases on local or remote given SQL instance except the ones specified in the exclude list.
 ## Every found database will be backed up as separate backup task.
 DATABASES_ALL=yes
-DATABASES_ALL_EXCLUDE_LIST="test;mysql"
+DATABASES_ALL_EXCLUDE_LIST="test;phplistdb_prod" #DEBUG
 ## Alternatively, if DATABASES_ALL=no, you can specify a list of databases to backup separated by spaces.
-DATABASES_LIST=""
+#DATABASES_LIST="somedatabase"
 
 ## Max backup execution time per Database task. Soft max exec time generates a warning only. Hard max exec time generates a warning and stops current backup task.
 ## If a task gets stopped, next one in the task list gets executed. Time is specified in seconds.
@@ -88,31 +91,24 @@ HARD_MAX_EXEC_TIME_DB_TASK=7200
 ## Generally, level 5 is a good compromise between cpu, memory hunger and compress ratio. Gzipped files are set to be rsyncable.
 COMPRESSION_LEVEL=3
 
-## SQL Dump compression should be done on remote side but can also be done locally to lower remote system usage (will take more bandwidth, check for ssh compression)
+###### FILES SPECIFIC OPTIONS
-COMPRESSION_REMOTE=yes
 
-## ---------- FILES BACKUP OPTIONS
+## File backups are divided in tasks. Every directory in DIRECTORY_LIST will be processed as a unique task.
+## Every subdirectory of each directory in RECURSIVE_DIRECTORY_LIST will be processed as a unique task.
+## Example: RECURSIVE_DIRECTORY_LIST="/home;/var" will create backup tasks tasks "/home/dir1, "/home/dir2", ... "/home/dirN", "/var/log", "/var/lib"... "/var/something".
+## You can exclude directories from the avove backup task creation, ex: avoid backing up "/home/dir2" by adding it to RECURSIVE_EXCLUDE_LIST.
 
-## Directories backup list. List of semicolon separated directories that will be backed up recursively. Every directory will be processed as one backup task.
+## Directories backup list. List of semicolon separated directories that will be backed up.
-DIRECTORIES_SIMPLE_LIST="/var/named;/var/lib"
+DIRECTORY_LIST="/var/named"
+RECURSIVE_DIRECTORY_LIST="/home"
+RECURSIVE_EXCLUDE_LIST="/home/backupuser;/home/lost+found"
 
-## There's a special backup schema in Obackup called 'backup task division'  which creates one backup task per level 1 subdirectory of a directory.
+## Rsync exclude patterns, used by simple and division lists, separated by semicolons
-## This is VERY useful to backup multiple virtualhosts as separate tasks without having to specify each one separately.
-## This may also be useful dividing big data directories in subdirectories tasks.
-
-## Directories backup task division backup: Semicolon separated directories of which every level 1 subdirectory will be backed up recursively as a separate backup task.
-## Example: "/home;/var" will create tasks "/home/dir1", "/home/dir2", ... "/home/dirN", "/var/log", "/var/lib"... "/var/whatever"
-DIRECTORIES_RECURSE_LIST="/home"
-## You may optionally exclude subdirectories from task division. On the above example you could exclude /home/dir2 by adding it to DIRECTORIES_RECURSE_EXCLUDE_LIST
-DIRECTORIES_RECURSE_EXCLUDE_LIST="/home/backupuser;/home/lost+found"
-
-## Rsync exclude patterns, used by simple and division lists
 RSYNC_EXCLUDE_PATTERN="*/tmp;*/ftp/www/cache/cachefs;*/sessions"
 
 ## File that contains the list of directories or files to exclude from sync on both sides. Leave this empty if you don't want to use an exclusion file.
 ## This file has to be in the same directory as the config file
 ## Paths are relative to sync dirs. One element per line.
-RSYNC_EXCLUDE_FROM=""
 #RSYNC_EXCLUDE_FROM="exclude.list"
 
 ## List separator char. You may set an alternative separator char for your directories lists above.
@@ -147,7 +143,13 @@ DELETE_VANISHED_FILES=no
 ## Use delta copy algortithm (usefull when local paths are network drives), defaults to yes
 DELTA_COPIES=yes
 
-## ---------- ALERT OPTIONS
+## Bandwidth limit Kbytes / second for file backups. Leave 0 to disable limitation.
+BANDWIDTH=0
+
+## Paranoia option. Don't change this unless you read the documentation.
+RSYNC_EXECUTABLE=rsync
+
+###### ALERT OPTIONS
 
 ## Alert email addresses separated by a space character
 DESTINATION_MAILS="your@mail.address"
@@ -158,7 +160,7 @@ SMTP_SERVER=smtp.your.isp.com
 SMTP_USER=
 SMTP_PASSWORD=
 
-## ---------- GENERAL BACKUP OPTIONS
+###### GENERAL BACKUP OPTIONS
 
 ## Max execution time of whole backup process. Soft max exec time generates a warning only.
 ## Hard max exec time generates a warning and stops the whole backup execution.
@@ -166,10 +168,11 @@ SOFT_MAX_EXEC_TIME_TOTAL=30000
 HARD_MAX_EXEC_TIME_TOTAL=36000
 
 ## Backup Rotation. You may rotate backups if you don't use snapshots on your backup server.
-ROTATE_BACKUPS=no
+ROTATE_SQL_BACKUPS=no
+ROTATE_FILE_BACKUPS=no
 ROTATE_COPIES=7
 
-## ---------- EXECUTION HOOKS
+###### EXECUTION HOOKS
 
 ## Commands can will be run before and / or after backup execution (remote execution will only happen if REMOTE_BACKUP is set).
 ## This is useful to make a snapshot before backing up data, or even handle snapshots of backed up data.

install.sh

@@ -1,25 +1,77 @@
 #!/usr/bin/env bash
 
-SCRIPT_BUILD=2015082501
+PROGRAM=obackup
+PROGRAM_BINARY=$PROGRAM".sh"
+PROGRAM_BATCH=$PROGRAM"-batch.sh"
+SCRIPT_BUILD=2015102701
 
-## Obackup install script
+## osync / obackup daemon install script
-## Tested on RHEL / CentOS 6 & 7
+## Tested on RHEL / CentOS 6 & 7 and Mint 17
 ## Please adapt this to fit your distro needs
 
-if [ "$(whoami)" != "root" ]
+CONF_DIR=/etc/$PROGRAM
-then
+BIN_DIR=/usr/local/bin
+SERVICE_DIR=/etc/init.d
+
+if [ "$(whoami)" != "root" ]; then
   echo "Must be run as root."
   exit 1
 fi
 
-mkdir /etc/obackup
+if [ ! -d "$CONF_DIR" ]; then
-cp ./host_backup.conf /etc/obackup/host_backup.conf.example
+	mkdir "$CONF_DIR"
-cp ./exclude.list.example /etc/obackup
+	if [ $? == 0 ]; then
-cp ./obackup.sh /usr/local/bin
+		echo "Created directory [$CONF_DIR]."
-cp ./obackup-batch.sh /usr/local/bin
+	else
+		echo "Cannot create directory [$CONF_DIR]."
+		exit 1
+	fi
+else
+	echo "Config directory [$CONF_DIR] exists."
+fi
+
+if [ -f ./sync.conf ]; then
+	cp ./sync.conf /etc/$PROGRAM/sync.conf.example
+fi
+
+if [ -f ./host_backup.conf ]; then
+	cp ./host_backup.conf /etc/$PROGRAM/host_backup.conf.example
+fi
+
+if [ -f ./exlude.list.example ]; then
+	cp ./exclude.list.example /etc/$PROGRAM
+fi
+
+cp ./$PROGRAM_BINARY "$BIN_DIR"
+if [ $? != 0 ]; then
+	echo "Cannot copy $PROGRAM_BINARY to [$BIN_DIR]."
+else
+	echo "Copied $PROGRAM_BINARY to [$BIN_DIR]."
+fi
+cp ./$PROGRAM_BATCH /usr/local/bin
+if [ $? != 0 ]; then
+	echo "Cannot copy $PROGRAM_BATCH to [$BIN_DIR]."
+else
+	echo "Copied $PROGRAM_BATCH to [$BIN_DIR]."
+fi
 cp ./ssh_filter.sh /usr/local/bin
-chmod 755 /usr/local/bin/obackup.sh
+if [ $? != 0 ]; then
-chmod 755 /usr/local/bin/obackup-batch.sh
+	echo "Cannot copy ssh_filter.sh to [$BIN_DIR]."
+else
+	echo "Copied ssh_filter.sh to [$BIN_DIR]."
+fi
+
+if [ -f ./osync-srv ]; then
+	cp ./osync-srv "$SERVICE_DIR"
+	if [ $? != 0 ]; then
+		echo "Cannot copy osync-srv to [$SERVICE_DIR]."
+	else
+		echo "Created osync-srv service in [$SERVICE_DIR]."
+		chmod 755 /etc/init.d/osync-srv
+	fi
+fi
+
+chmod 755 /usr/local/bin/$PROGRAM_BINARY
+chmod 755 /usr/local/bin/$PROGRAM_BATCH
 chmod 755 /usr/local/bin/ssh_filter.sh
 chown root:root /usr/local/bin/ssh_filter.sh
-

obackup-batch.sh

@@ -1,15 +1,15 @@
 #!/usr/bin/env bash
+SUBPROGRAM=obackup
+PROGRAM="$SUBPROGRAM-batch" # Batch program to run osync / obackup instances sequentially and rerun failed ones
+AUTHOR="(L) 2013-2015 by Orsiris \"Ozy\" de Jong"
+CONTACT="http://www.netpower.fr - ozy@netpower.fr"
+PROGRAM_BUILD=2015103001
 
-PROGRAM="Obackup-batch" # Batch program to run obackup instances sequentially and rerun failed ones
+## Runs an osync /obackup instance for every conf file found
-AUTHOR="(L) 2013-2014 by Orsiris \"Ozy\" de Jong"
-CONTACT="http://www.netpower.fr/obackup - ozy@netpower.fr"
-PROGRAM_BUILD=2508201501
-
-## Runs an obackup instance for every conf file found
 ## If an instance fails, run it again if time permits
 
-## Configuration file path. The path where all the obackup conf files are, usually /etc/obackup
+## Configuration file path. The path where all the osync / obackup conf files are, usually /etc/osync or /etc/obackup
-CONF_FILE_PATH=/etc/obackup
+CONF_FILE_PATH=/etc/$SUBPROGRAM
 
 ## If maximum execution time is not reached, failed instances will be rerun. Max exec time is in seconds. Example is set to 10 hours.
 MAX_EXECUTION_TIME=36000
@@ -18,60 +18,76 @@ MAX_EXECUTION_TIME=36000
 MAX_RERUNS=3
 
 ## Log file path
-if [ -w /var/log ]
+if [ -w /var/log ]; then
-then
+	LOG_FILE=/var/log/$SUBPROGRAM-batch.log
-        LOG_FILE=/var/log/obackup-batch.log
 else
-        LOG_FILE=./obackup-batch.log
+	LOG_FILE=./$SUBPROGRAM-batch.log
 fi
 
 # No need to edit under this line ##############################################################
 
-function Log
+function _logger {
-{
+	local value="${1}" # What to log
-        prefix="TIME: $SECONDS - "
+	echo -e "$value" >> "$LOG_FILE"
-        echo -e "$prefix$1" >> "$LOG_FILE"
 
-        if [ $silent -eq 0 ]
+	if [ $_SILENT -eq 0 ]; then
-        then
+		echo -e "$value"
-                echo -e "$prefix$1"
+	fi
-        fi
 }
 
-function CheckEnvironment
+function Logger {
-{
+	local value="${1}" # What to log
-        ## Obackup executable full path can be set here if it cannot be found on the system
+	local level="${2}" # Log level: DEBUG, NOTICE, WARN, ERROR, CRITIAL
-        if ! type -p obackup.sh > /dev/null 2>&1
+
-        then
+	prefix="$(date) - "
-                if [ -f /usr/local/bin/obackup.sh ]
+
-                then
+	if [ "$level" == "CRITICAL" ]; then
-                        OBACKUP_EXECUTABLE=/usr/local/bin/obackup.sh
+		_logger "$prefix\e[41m$value\e[0m"
-                elif [ -f ./obackup.sh ]
+		ERROR_ALERT=1
-                then
+	elif [ "$level" == "ERROR" ]; then
-                        OBACKUP_EXECUTABLE=./obackup.sh
+		_logger "$prefix\e[91m$value\e[0m"
-                else
+		ERROR_ALERT=1
-                        Log "Could not find obackup.sh"
+	elif [ "$level" == "WARN" ]; then
-                        exit 1
+		_logger "$prefix\e[93m$value\e[0m"
-                fi
+	elif [ "$level" == "NOTICE" ]; then
-        else
+		_logger "$prefix$value"
-                OBACKUP_EXECUTABLE=$(type -p obackup.sh)
+	elif [ "$level" == "DEBUG" ]; then
-        fi
+		if [ "$DEBUG" == "yes" ]; then
+			_logger "$prefix$value"
+		fi
+	else
+		_logger "\e[41mLogger function called without proper loglevel.\e[0m"
+		_logger "$prefix$value"
+	fi
+}
+
+function CheckEnvironment {
+	## osync / obackup executable full path can be set here if it cannot be found on the system
+	if ! type -p $SUBPROGRAM.sh > /dev/null 2>&1
+	then
+		if [ -f /usr/local/bin/$SUBPROGRAM.sh ]
+		then
+			SUBPROGRAM_EXECUTABLE=/usr/local/bin/$SUBPROGRAM.sh
+		else
+			Logger "Could not find $SUBPROGRAM.sh" "CRITICAL"
+			exit 1
+		fi
+	else
+		SUBPROGRAM_EXECUTABLE=$(type -p $SUBPROGRAM.sh)
+	fi
 
 	## Check for CONF_FILE_PATH
-        if [ ! -d "$CONF_FILE_PATH" ]
+	if [ ! -d "$CONF_FILE_PATH" ]; then
-        then
+		Logger "Cannot find conf file path $CONF_FILE_PATH" "CRITICAL"
-                Log "Cannot find conf file path $CONF_FILE_PATH"
+		Usage
-                Usage
+	fi
-        fi
 }
 
-function Batch
+function Batch {
-{
 	## Get list of .conf files
 	for i in $(ls $CONF_FILE_PATH/*.conf)
 	do
-		if [ "$RUN" == "" ]
+		if [ "$RUN" == "" ]; then
-		then
 			RUN="$i"
 		else
 			RUN=$RUN" $i"
@@ -81,21 +97,20 @@ function Batch
 	RERUNS=0
 	while ([ $MAX_EXECUTION_TIME -gt $SECONDS ] || [ $MAX_EXECUTION_TIME -eq 0 ]) && [ "$RUN" != "" ] && [ $MAX_RERUNS -gt $RERUNS ]
 	do
-		Log "Obackup instances will be run for: $RUN"
+		Logger "$SUBPROGRAM instances will be run for: $RUN" "NOTICE"
 		for i in $RUN
 		do
-			$OBACKUP_EXECUTABLE "$i" $opts
+			$SUBPROGRAM_EXECUTABLE "$i" $opts &
-			if [ $? != 0 ]
+			wait $!
-			then
+			if [ $? != 0 ]; then
-				Log "Run instance $(basename $i) failed"
+				Logger "Run instance $(basename $i) failed" "ERROR"
-				if [ "RUN_AGAIN" == "" ]
+				if [ "RUN_AGAIN" == "" ]; then
-				then
 					RUN_AGAIN="$i"
 				else
 					RUN_AGAIN=$RUN_AGAIN" $i"
 				fi
 			else
-				Log "Run instance $(basename $i) succeed."
+				Logger "Run instance $(basename $i) succeed." "NOTICE"
 			fi
 		done
 		RUN="$RUN_AGAIN"
@@ -104,44 +119,43 @@ function Batch
 	done
 }
 
-function Usage
+function Usage {
-{
+	echo "$PROGRAM $PROGRAM_BUILD"
-        echo "$PROGRAM $PROGRAM_BUILD"
+	echo $AUTHOR
-        echo $AUTHOR
+	echo $CONTACT
-        echo $CONTACT
+	echo ""
-        echo ""
+	echo "Batch script to sequentially run osync or obackup instances and rerun failed ones."
-        echo "Batch script to sequentially run obackup instances and rerun failed ones."
+	echo "Usage: $SUBPROGRAM-batch.sh [OPTIONS]"
-        echo "Usage: obackup-batch.sh [OPTIONS]"
+	echo ""
-        echo ""
+	echo "[OPTIONS]"
-        echo "[OPTIONS]"
+	echo "--path=/path/to/conf      Path to osync / obackup conf files, defaults to /etc/osync or /etc/obackup"
-	echo "--path=/path/to/conf      Path to obackup conf files, defaults to /etc/obackup"
 	echo "--max-reruns=X            Number of runs  max for failed instances, (defaults to 3)"
-	echo "--max-exec-time=X         Retry failed instances only if max execution time not reached (defaults to 36000 seconds)"
+	echo "--max-exec-time=X         Retry failed instances only if max execution time not reached (defaults to 36000 seconds). Set to 0 to bypass execution time check."
-	echo "--no-maxtime		Run obackup without honoring conf file defined timeouts"
+	echo "--no-maxtime		Run osync / obackup without honoring conf file defined timeouts"
-        echo "--dry                     Will run obackup without actually doing anything; just testing"
+	echo "--dry                     Will run osync / obackup without actually doing anything; just testing"
-        echo "--silent                  Will run obackup without any output to stdout, used for cron jobs"
+	echo "--silent                  Will run osync / obackup without any output to stdout, used for cron jobs"
-        echo "--verbose                 Increases output"
+	echo "--verbose                 Increases output"
-        exit 128
+	exit 128
 }
 
-silent=0
+_SILENT=0
-dry=0
+_DRY=0
-verbose=0
+_VERBOSE=0
 opts=""
 for i in "$@"
 do
-        case $i in
+	case $i in
-                --silent)
+		--silent)
-                silent=1
+		_SILENT=1
 		opts=$opts" --silent"
-                ;;
+		;;
-                --dry)
+		--dry)
-                dry=1
+		_DRY=1
 		opts=$opts" --dry"
-                ;;
+		;;
-                --verbose)
+		--verbose)
-                verbose=1
+		_VERBOSE=1
-                opts=$opts" --verbose"
+		opts=$opts" --verbose"
 		;;
 		--no-maxtime)
 		opts=$opts" --no-maxtime"
@@ -159,12 +173,12 @@ do
 		Usage
 		;;
 		*)
-		Log "Unknown param '$i'"
+		Logger "Unknown param '$i'" "CRITICAL"
 		Usage
 		;;
 	esac
 done
 
 CheckEnvironment
-Log "$(date) Obackup batch run"
+Logger "$(date) $SUBPROGRAM batch run" "NOTICE"
 Batch

ssh_filter.sh

@@ -1,6 +1,6 @@
-#!/bin/bash
+#!/usr/bin/env bash
 
-##### Osync ssh command filter build 2015070203
+##### osync / obackup ssh command filter build 2015102701
 ##### This script should be located in /usr/local/bin in the remote system to sync / backup
 ##### It will filter the commands that can be run remotely via ssh.
 ##### Please chmod 755 and chown root:root this file
@@ -19,77 +19,77 @@ CMD3=
 
 LOG_FILE=~/.ssh/ssh_filter.log
 
-function Log
+function Log {
-{
 	DATE=$(date)
 	echo "$DATE - $1" >> $LOG_FILE
 }
 
-function Go
+function Go {
-{
 	eval $SSH_ORIGINAL_COMMAND
 }
 
 case ${SSH_ORIGINAL_COMMAND%% *} in
 	"$RSYNC_EXECUTABLE")
 	Go ;;
+	"echo")
+	Go ;;
 	"find")
 	Go ;;
 	"du")
 	Go ;;
-	"mysql")
+	"mkdir")
 	Go ;;
-	"mysqldump")
+	"rm")
+	Go ;;
+	"df")
+	Go ;;
+	"mv")
 	Go ;;
 	"$CMD1")
-	if [ "$CMD1" != "" ]
+	if [ "$CMD1" != "" ]; then
-	then
 		Go ;;
 	fi
 	"$CMD2")
-	if [ "$CMD2" != "" ]
+	if [ "$CMD2" != "" ]; then
-	then
 		Go ;;
 	fi
 	"$CMD3")
-	if [ "$CMD3" != "" ]
+	if [ "$CMD3" != "" ]; then
-	then
 		Go ;;
 	fi
 	"sudo")
-	if [ "$SUDO_EXEC" == "yes" ]
+	if [ "$SUDO_EXEC" == "yes" ]; then
-	then
+		if [[ "$SSH_ORIGINAL_COMMAND" == "sudo $RSYNC_EXECUTABLE"* ]]; then
-		if [[ "$SSH_ORIGINAL_COMMAND" == "sudo $RSYNC_EXECUTABLE"* ]]
+			Go
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo du"* ]]; then
+			Go
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo find"* ]]; then
+			Go
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo mkdir"* ]]
 		then
 			Go
-		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo du"* ]]
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo rm"* ]]
 		then
 			Go
-		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo find"* ]]
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo echo"* ]]
 		then
 			Go
-                elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo mysql"* ]]
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo df"* ]]
-                then
-                        Go
-                elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo mysqldump"* ]]
-                then
-                        Go
-		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo $CMD1"* ]]
 		then
-			if [ "$CMD1" != "" ]
+			Go
-			then
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo mv"* ]]
+		then
+			Go
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo $CMD1"* ]]; then
+			if [ "$CMD1" != "" ]; then
 			Go
 			fi
-		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo $CMD2"* ]]
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo $CMD2"* ]]; then
-		then
+			if [ "$CMD2" != "" ]; then
-			if [ "$CMD2" != "" ]
-			then
 			Go
 			fi
-		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo $CMD3"* ]]
+		elif [[ "$SSH_ORIGINAL_COMMAND" == "sudo $CMD3"* ]]; then
-		then
+			if [ "$CMD3" != "" ]; then
-			if [ "$CMD3" != "" ]
-			then
 			Go
 			fi
 		else