obackup/CHANGELOG.md

KNOWN ISSUES
------------

- Backup size check does not honor rsync exclude patterns
- Encryption does not honor rsync exclude patterns
- Bandwidth parameter is ignored for SQL backups
- Missing symlink support when run from MSYS environment

CHANGELOG
---------

dd Mmm YYYY: obackup v2.1 RC2 released
--------------------------------------

- Added a default required config file revision
	- ! Update script updated accordingly
- Updated ofunctions to use booleans instead of yes/no syntax which still works
- Fixed verbose rsync output not working
- Fixed a potential bash buffer overflow when very large file lists are logged

03 Jan 2019: obackup v2.1 RC1 released
--------------------------------------

- File backup rotation will always make copies instead of moving files, in order to let rsync do deltas even in local backup scheme
- Fixed non recursive backups being recursive (bug introduced in a8b6bcb)
- Fixed multiple trailing slashes when backing-up '/'
- Updated ofunctions
- Upgraded shunit2 test framework to v2.1.8pre (git commit 07bb329)
- Minor fixes

20 Jun 2017: obackup v2.1 beta3 released
----------------------------------------

- Fixed regression where some commandline arguments weren't honored anymore since 2.1 beta1 (--delete, --stats, --dontgetsize)
- Fixed commandline arguments aren't checked against valid list

14 Mar 2017: obackup v2.1 beta2 released
----------------------------------------

- Fixed remote commands can be run on local runs and obviously fail
- Uninstall leaves ssh_filter if needed by other programs
- Logger now obfuscates _REMOTE_TOKEN
- Improved sudo privilege run
- Brand new ssh filter from osync project
- Better installer with --remove option from osync project
- Updated ofunctions from osync project
	- Fixes UTF-8 escaped characters in log files due to LC_ALL=C
	- Optional MAIL_BODY_CHARSET so destination mails aren't sent as UTF-8 anymore depending on systems
- Minor fixes

04 Jan 2017: obackup v2.1 beta1 released
----------------------------------------

- Fixed wrong file size fetched remotely since v2.1 rewrite
- Fixed missing databases in manual list fails to trigger an alert
- Improved support for GPG ver >= 2.1
- Added encryption / decryption parallel execution support
- Improved compatibility for RotateCopies
- Unit tests now run on CentOS 5,6
- Added optional rsync arguments configuration value
- Forcec bash usage on remote connections in order to be FreeBSD 11 compatible
- Spinner is less prone to move logging on screen
- Fixed another random error involving warns and errors triggered by earlier runs with same PID flag files
- Adde more preflight checks (pgrep presence)
- Added --no-prefix, --error-only and --summary switches
- Updated installer from osync
- Updated merge.sh script to handle includes
- Improved remote logging
- Simplified osync-batch runner (internally and for user)
	- Better filename handling
	- Easier to read log output
        - Always passes --silent to obackup
        - All options that do not belong to obackup-batch are automatically passed to obackup
- Improved installer OS detection
- Fixed upgrade script cannot update header on BSD / MacOS X
- Fixed SendEmail function on MacOS X
- Fixed MAX_SOFT_EXEC_TIME_PER_XX_TASK not enforced bug introduced with newer ofunctions from v2.1
- PRESERVE_ACL and PRESERVE_XATTR are ignored when local or remote OS is MacOS or msys or Cygwin
- Fixed PRESERVE_EXECUTABILITY was ommited volontary on MacOS X because of rsync syntax
- merge.sh is now BSD and Mac compatible
- Unit tests are now BSD and Mac compatible
- Local runs should not check for remote connectivity
- Fixed error alerts cannot be triggered from subprocesses
- Fixed error flags 
- Faster remote OS detection
- Added busybox (and Android Termux) support
	- More portable file size functions
	- More portable compression program commands
	- More paranoia checks
	- Added busybox sendmail support
	- Added tls and ssl support for sendmail
- Added ssh password file support
- Added unit tests
	- Added basic unit tests for all three operation modes
	- Added process management function tests
	- Added file rotation tests
	- Added upgrade script test
	- Added encryption tests
	- Added missing files / databases test
	- Added timed execution tests
- Implemented backup encryption using GPG (see documentation for advantages and caveats)
	- Backup encrypted but still use differential engine :)
- Database backup improvements
	- Added mysqldump options to config file
- Improved unit tests
- Added more preflight checks
- Logs sent by mail are easier to read
	- Better subject (currently running or finished run)
	- Fixed bogus double log sent in alert mails
	- Only current run log is now sent
	- Alert sending is now triggered after last action
- Made unix signals posix compliant
- Improved upgrade script
	- Upgrade script now updates header
	- Can add any missing value now
	- Added encrpytion support
- Fixed problem with spaces in directories to backup (again !)
- Added options to ignore permissions, ownership and groups
- Improved batch runner
	- Batch runner works for directories and direct paths
	- Fixed batch runner does not rerun obackup on warnings only
	- Code compliance
	- More clear semantic
- Made keep logging value configurable and not mandatory
- Fixed handling of processes in uninterruptible sleep state
- Code cleanup
- Refactored waiting functions
- Fixed double RunAfterHook launch

06 Aug 2016: obackup v2.0 released
----------------------------------

- Made logging begin before remote checks for sanity purposes
- RunAfterCommands can get executed when trapquit
- Improved process killing and process time control
- Added optional statistics for installer
- Added an option to ignore knownhosts for ssh connections (use with caution, this can lead to a security issue)
- Improved mail fallback
- More logging enhancements
- Improved upgrade script
- Revamped rsync patterns to allow include and exclude patterns
- Better SQL and file backup task separation (rotate copies and warnings are defined for sql and/or file)
- Added reverse backup, now backups can be local, pushed or pulled to or from a remote system
- Better fallback for SendAlert even if disk full
- Added an alert email sent on warnings while backup script is running
- Way better logging of errors in _GetDirectoriesSizeX, _BackupDatabaseX, _CreateStorageDirectoriesX
- Added bogus config file checks & environment checks
- Full code refactoring to use local and remote code once
- Fully merged codebase with osync
	- Added (much) more verbose debugging (and possibility to remove debug code to gain speed)
	- Replace child_pid by $? directly, add a better sub process killer in TrapQuit
	- Added some automatic checks in code, for _DEBUG mode (and _PARANOIA_DEBUG now)
	- Improved Logging
	- Updated obackup to be fully compliant with coding style
- Fixed creation of bogus subdirectories in some cases
- A long list of minor improvements and bug fixes

v0-1.x - Jan 2013 - Oct 2015
----------------------------

- New function to kill child processes
- Fixed no_maxtime not honored
- Improved some logging, also added highlighting to stdout errors
- Backported some fixes from Osync
	- Small improvements on install script
	- Copy ssh_filter.sh from osync project
	- Small improvements in obackup-batch.sh time management
- Quick and dirty hack to get the full last run log in SendAlert email
- Added detection of obackup.sh script in obackup-batch.sh to overcome mising path in crontab
- Moved command line arguments after config file load for allowing command line overrides
- Added a config file option equivalent to --dontgetsize
- Added basic install script from osync project
- Added obackup-batch.sh from osync project to rerun failed backups in row
- Delta copy algorithm is now used even for local copies (usefull for network drives), this can be overriden in config file
- Added --dontgetsize parameter to backup huge systems immediatly
- Fixed multiple keep logging messages since sleep time between commands has been lowered under a second
- Create local subdirectories if not exist before running rsync (rsync doesn't handle mkdir -p)
- Backported some fixes from Osync
	- Lowered sleep time between commands
	- Lowered debug sleep times
	- Fixed a bug with exclude pattern globbing preventing multiple exludes
	- Lowered default compression level for email alerts (for low end systems)
- Prevent exclude pattern globbing before the pattern reaches the rsync cmd
- Fixed some typos with ported code from osync rendering stats and partial downloads unusable
- Added delete on destination option for files that vanished from source
- Fixed ignoring compression level in conf file
- Added experimental partial downloads support for rsync so big files can be resumed on slow links
- Fixed dry mode sql backup output
- Prevented triggering TrapError if there are no child processes to terminate on TrapQuit
- Improved mysql debug logs
- Prevent creation of backup-id less log file when DEBUG is set
- WARNING: Default behavior is now to copy the referrent files and directories from symlinks (this can reach files outside the backup root)
- Recursive directory search now includes symlinks (find -L option. -type d cannot be replaced by -xtype d because of portability issues with BSD)
- Dry mode does not create target directories anymore
- Dry mode also tries mysqldumps now (check for error messages being logged)
- Added experimental partial download support
- Added Rsync exclude files suppport from osync
- Fixed another issue with existing symlinks to directories on target on non recursive backups
- Fixed remaining rsync -E option preventing obackup to work correctly on MacOS X
- Fixed an issue with existing symlinks to directories on target
- Prevent changed IFS to make ping commands fail
- Added RotateCopies execution time (spinner support)
- redirect stderr for mysqldump to catch problems
- Moved msys specific code to Init(Local|Remote)OSSettings except in TrapQuit that needs to work at any moment
- Added support for multithreaded gzip (if pigz is installed)
- Merged back changes from osync codebase
	- Enhanced debugging
	- Added language agnostic system command output
	- Enhanced log sending
	- Better handling of OS specific commands
	- Improved WaitForTaskCompletion when DEBUG enabled or SILENT enabled
	- Enhanced OS detection
- More correct error message on remote connection failure
- Gzipped logs are now deleted once sent
- Fixed some typos (thanks to Pavel Kiryukhin)
- Improved OS detection and added prelimnary MacOS X support
- Improved execution hook logs
- Improved RunLocalCommand execution hook

02 Nov. 2013: obackup v1.84RC3 released
---------------------------------------

- Updated documentation
- Minor rewrites in recursive backup code
- Added base directory files backup for recursive directories backup
- Minor improvements on permission checks
- Added local and remote OS detection
- Fixed ping arguments for FreeBSD compatibility
- Added MSYS (MinGW minimal system) bash compatibility under Windows
	- Added check for /var/log directory
	- Added check for shared memory directory
	- Added alternative way to kill child processes for other OSes and especially for MSYS (which is a very odd way)
	- Added Sendemail.exe support for windows Alerting
	- Replaced which commend by type -p, as it is more portable
	- Added support for ping.exe from windows
	- Forced usage of MSYS find instead of Windows' find.exe
	- Added an optionnal remote rsync executable path parameter
	- Made ListDatabases and ListDirectories Msys friendly
- Fixed loop problems in RotateBackups and ListDatabases (depending on IFS environment)
- Fixed an error in CheckSpaceRequirements not setting required space to zero if file / sql backup is disabled
- Fixed an issue with CheckConnectivity3rdPartyHosts
- Added option to stop execution on failed command execution
- Improved forced quit command by killing all child processes
- Before / After commands are now ignored on dryruns
- Improved verbose output
- Improved dryrun output
- Improved remote connecivity detection 
- Fixed a typo in configuration file

18 Aug. 2013: obackup v1.84RC2 released
---------------------------------------

- Added possibility to change default logfile
- Simplified dryrun (removed dryrun function and merged it with main function)
- Simplified Init function
- Added --stat switch to rsync execution
- Added bandwidth limit
- Added --no-maxtime switch
- Fixed LoadConfigFile function will not warn on wrong config file
- More code cleanup
- Added --verbose switch (will add databases list,  rsync commands, and file backup list)
- Improved task execution checks and more code cleanup
- Fixed CleanUp function if DEBUG=yes, also function is now launched from TrapQuit

16 Jul. 2013: obackup v1.84RC1 released
---------------------------------------

- Code cleanup
- Uploaded first documentation
- Fixed an issue with RotateBackups
- Updated obackup to log failed ssh command results
- Updated ssh command filter to log failed commands
- Updated ssh command filter to accept personalized commands
- 23 Jun. 2013: v1.84 RC1 approaching
- Added ssh commands filter, updated documentation
- Rewrote local space check function
- Added ability to run another executable than rsync (see documentation on sudo execution)
- Added some Rsync argument parameters (preserve ACL, Xattr, and stream compression)
- Internal hook execution logic revised
- Updated WaitForTaskCompletition function to handle skipping alerts
- Updated command line argument --silent processing
- Added remote before and after command execution hook
- Added local before and after command execution hook

14 Jun 2013
-----------

- Initial public release, fully functionnal