Today, customers with large numbers of files (typically in the millions) can experience very long incremental backup times. Two factors contributing to the delayed backup times include:
• | Factor 1 – The Windows agent scans all files/directories on the specified volumes to determine modification times. Files with modification times more recent than the last successful master are included in the incremental backup. |
• | Factor 2 – The Windows agent also looks at the ‘archive’ property on each file whose modification time does not meet the first criteria. If the property is set, then the file is included in the backup. |
Factor 1 accounts for a large percentage of the time spent performing the backup, while Factor 2 contributes more processing overhead from the mechanics of examining the file properties. In addition, Factor 2 helps to inflate the size of the backup by including files that do not meet the modification criteria. Additionally, files with the archive property set should not automatically be included in incremental backups.
Use of the change journal can eliminate much of the overhead contributed by these factors. The change journal is a record of all changes to any file(s) on a given volume. When a change journal is created, Windows begins to log changes immediately to that journal without requiring a reboot of the server. Windows logs all file changes on a given volume along with the nature and time of the change.
During an incremental backup, the Unitrends agent queries the change journal to discover the changes made to files on the volume. It queries the data logged for each change to determine if the time of the change qualifies the data for back up. This is done by comparing the modified data to the time of the last successful master backup. Since journal records are kept only for changes to files or directories, determining which files to back up on a volume requires just a fraction of the time needed for the traditional volume scan. As the number of files on a volume increases, the benefit of using the change journal also increases.
The change journal feature is transparent. There are no visible configuration or setting options on the backup system or on Unitrends local agent interface. By default, the agent prefers to use the change journal during incremental backups. If the journal cannot be used, the agent uses the volume-scanning method to produce the list of files to back up.
See the following topics for details:
When a master backup is performed on the Unitrends agent, the system is scanned for an existing change journal on each volume. If a change journal does not exist on a volume, the agent creates one. If a journal does exist, the agent insures that the size of the journal meets the minimum size required by enlarging the journal if necessary. The minimum size required for a change journal is 500MB. After the journal creation/discovery is complete, the agent “registers” each journal by creating entries in the system’s registry. Subsequent incremental backups query these entries. The registry entries exist outside of the current Unitrends registry entries to insure that they are not removed following agent software updates or software removal. The agent records a registry entry for each volume containing a change journal. The registry entry contains the following:
• | Unique ID given to the change journal |
• | Sequential number of the first entry in the journal |
When an incremental backup is performed, the system is scanned for an existing change journal on each volume that it intends to back up. If a journal exists, the agent queries the registry for the journal’s ID and starting sequence number. This information will have been entered there by a previous master backup. If the entries exist, then the agent performs the following checks:
• | A check to determine if the journal ID in the registry matches the ID of the volume’s current journal. If the IDs do not match, this indicates that the journal that existed during the master backup was deleted and that a new journal was created. |
• | A check to determine if the starting sequence number in the registry matches that of the volume’s current journal. If these numbers do no match, this indicates that the journal was filled to capacity with entries and has ‘wrapped’ around to the beginning. In this case, there will be file modifications that were made to the volume that are not recorded in the journal. |
If both these checks succeed, then the agent uses the change journal to determine which files to include in the backup. If one of these checks fails, the agent does not use the journal and reverts to the volume scanning method. In this case, the agent includes some information in the backup output to warn the user that a new master backup must be completed in order to use the journal for subsequent incremental backups.
If use of the change journal is not desired, it can be disabled by modifying the following entry in the agent’s master.ini file. The file is located on the Windows client in the <agentInstallPath>\PCBP directory (typically C:\PCBP).
UseChangeJournal=False
The minimum change journal size maintained by the Unitrends’ agent is 500MB. This size is configurable by adding an entry into the agent’s master.ini file. The size should be indicated in MB.
ChangeJournalSizeMB=500
The size of the change journal can be enlarged by creating the above entry and setting the value to a number larger than 500.
Warning! The Microsoft best practices for use of change journal recommends that, once created, the change journal size should not be reduced, nor should the change journal be deleted. Likewise, the Unitrends agent cannot guarantee successful backup and restore operations in an environment where the size of the change journal has been reduced. Please understand that reducing the size of the change journal may result in corrupt backup operations, possibly causing failure to restore.
Each time that a master backup completes, the change journal for each volume is inspected and enlarged if the size indicated in the master.ini file is larger than the size of the actual journal.
The change journal wrap condition causes a delay in the time it takes for incremental backups to complete. If incremental backups begin to take longer than expected, this may be a result of the change journal wrap condition. When backing up servers that are configured as domain controllers, the change journal wrap condition may occur more frequently.
The change journal wrap condition is not an error and will not interfere with capturing a successful backup. However, because the system must do a full filesystem scan, this does increase the likelihood of a timeout occurring that fails the backup. In the event that the change journal wrap condition occurs, expect to see the following message in the backup summary for the specific backup operation:
The change journal contains records of changes to files on a particular volume. There are two types of data changes that do not modify file content but are tracked by the journal:
• | Auxiliary Data Change – An operation adds a private data stream to a file or directory. An example might be a virus detector adding checksum information. As the virus detector modifies the item, the system generates change journal records. This type of file change indicates that the modifications did not change the application data. |
• | Basic Information Change – A user has changed one or more file or directory attributes (i.e. read-only, hidden, system, archive, or sparse), or one or more time stamps. |
When the agent encounters an ‘Auxiliary Data Change’ record, the default behavior is to ignore it and not include the file in the backup.
When the agent encounters a ‘Basic Information Change’ record, the default behavior is to include the modified file in the backup.
In both cases, if the default behavior is not the desired behavior, modify the master.ini file to override the existing settings:
ChgJournalBackupAuxChg=1
Adding this entry forces the agent to include files in the backup when an ‘Auxiliary Data Change’ is found.
ChgJournalBackupPropChg=0
Adding this entry forces the agent to ignore files whose only change was a property change.
Change journals are created and managed on a volume-by-volume basis. If a system contains multiple volumes and a subset of the volumes cannot support use of the change journal, the volumes are backed up using the volume scanning method. Volumes that support change journal are backed up using the change journal.
Change journals can be created and accessed on local NTFS volumes only. If NTFS file systems are mounted from remote servers, the incremental backups that include the mounted volumes do not use the change journal for those volumes.
For example:
File Server ‘ServerA ‘shares ‘DirectoryA’ for anyone to mount.
Workstation ‘UserA’ maps ‘DirectoryA’ as a local drive and then adds/change files.
If an incremental backup of ‘ServerA’ is performed: ‘UserA’ changes to ‘DirectoryA’ will be seen and backed up using the change journal.
If an incremental backup of ‘UserA’ is performed using the change journal: ‘UserA’ changes to ‘DirectoryA’ will not be backed up using the change journal.