DPM Replica Is Inconsistent

I just completed a new install of DPM for a client on a new server. The number of servers is fairly small. So, it wasn't worthwhile to migrate the existing backups to the new server. Instead, I just did a new install of DPM and reinstalled the DPM client software to point them at the new server.

After doing the initial backups, all looked good except for a generation 2 virtual machine running Windows Server 2012 R2. This server has the error Replica is inconsistent for Bare Metal Recovery and System State. All other data backed up fine.

Most of the problems I've ever had with DPM were due to Windows Backup not being installed on the server being backed up. Usually, installing that feature fixes any issues. However, in this case it was already installed. Which makes sense because the servers were being backed up successfully by the other DPM server.

The application event log gave me:

Event ID: 517, Error
The backup operation that started at DateAndTime has failed with the following error code '0x807800C5' (There was a failure in preparing the backup image of one of the volume in the backup set.)

The Windows\Backup event log gave me the same information plus some details in the XML view. One of the details was a path to the error log in C:\Windows\Logs\WindowsServerBackup.

Browsing to this location I found:

C:\Windows\Logs\WindowsServerBackup\Backup_Error-DateTime.txt

There were several instances of this file from running overnight. Each file had an error, but there were two possible errors.

Backup of volume \\?\Volume{GUID}\ has failed. The system cannot find the file specified.
Backup of volume \\?\Volume{GUID}\ has failed. The mounted backup volume is inaccessible. Please retry the operation.

This provided me with a volume GUID that I could use to track down the volume that was causing the issue. To see the volume GUIDs, I used the following PowerShell command:

Get-Volume | FL

From this information I could see that the volume causing the problem was a 300MB Recovery volume created automatically during installation. After doing some searching, I determined that the issue was caused by this volume having insufficient space to do a VSS snapshot during backup. In the 300 MB volume about 55MB was free.

There are three possible fixes for this issue:

• Disable WinRE. When you disable WinRE by running reagentc.exe /disable then the backup does not create a snapshot of the recovery partition. It is backed up, but a snapshot is not used. This is fine if you want to leave the recovery partition disabled.
• Move VSS storage to another drive. If you configure VSS to store snapshot changes to another volume such as C: then the backup completes properly. This is simple and effective. It takes little space on the other volume. So, there is no negative impact on storage space. The required command: vssadmin add shadowstorage /for=\\?\Volume{GUID}\ /on=c: /maxsize=500MB
• Create larger volume for WinRE. You can create a new larger volume for WinRE and reinstall the WinRE files on that partition. While this initially seems like the graceful choice, it is the hardest to implement. You need to have sufficient unallocated disk space. I personally wouldn't recommend it.

Additional resources:

• Discussion about this issue and detailed instructions. Mike Jacquet from Microsoft provides solid information about the cause and potential solutions.
• Microsoft KB that describes how to create a larger WinRE volume.