Avamar Exchange 2013 DAG backup fails – Waiting for VSS

Today I have been troubleshooting Avamar backups of an Exchange 2013 DAG (Database Availability Groups) setup. The Avamar job would start but no data would be backed up and the job would fail within 15 minutes. Initially VSS was suspected to be the culprit but additional troubleshooting revealed some strange behavior of the Avamar client in which the individual plugins would keep waiting for a status message that would never arrive.

The setup is simple: three Windows 2008 R2 SP1 servers, Exchange 2013 Update 5 with Database Availability Groups and Avamar 7.0.2 as the backup software and target. The error that pops up looks like this:

Avamar Exchange 2013 DAG error

Continue reading “Avamar Exchange 2013 DAG backup fails – Waiting for VSS”

Brocade reset to factory default settings (or just brick it)

A pile of decommissioned Brocade switches. Or now: doorsteps.Last weekend we replaced six old Brocade SAN switches with brand new Cisco MDS 9148 switches. Everything went according to plan with no disruption to the rest of the infrastructure. I was however stuck with a bunch of old Brocade 4900 switches ready to be decommissioned. Performing a Brocade reset to factory default settings proved to be a bit of a challenge though…

If you ask Google how to perform a Brocade reset to factory default settings, you’ll find a lot of commands. One command removes the zoning, another command removes a different part of the config, a third command replaces some config values with the default settings. However, none of these commands reset the IP configuration, user passwords or switch name. Which is kind of awkward since that’s THE part of the switch config you wouldn’t want to become public domain…

Continue reading “Brocade reset to factory default settings (or just brick it)”

CLARiiON CX3 vault drive failure

Yesterday I received a call about a drive failure in a CLARiiON CX3-80 storage array. Since every system has at least a couple of hot spares configured, usually this does not pose a problem. But this drive failure happened on drive position 0_0_0: a vault drive failure.

The vault drives host (amongst others) the operating system and configuration of the storage array. They are also used to destage the write cache in case of an event that might threaten data integrity.

Continue reading “CLARiiON CX3 vault drive failure”