For space reasons, this text is another bit that was cut out of my Exchange 2013 Inside Out: Mailbox and High Availability book. FWIW, here it is…
Imagine my annoyance when I ran the Get-MailboxDatabaseCopyStatus cmdlet on a test Exchange 2013 server and found that three of the databases reported a “Failed and Suspended” status for their content index. Despite the rumors to the contrary, I don’t spend my days idly running commands against Exchange to see what happens. I’d been playing with health sets and health reports in an effort to understand these concepts better and was provoked to finding out why my server was deemed unhealthy for data protection when I ran the command to extract the health report based on the data protection health set:
Get-HealthReport –Identity ExServer1 –HealthSet DataProtection
The content index is necessary to enable fast client searches so it is something to be concerned about. Failed and suspended means that Exchange hasn’t been able to fix any problems that it might encounter with the content indexes during normal operations and a reseed is necessary. I wasn’t all that worried in this instance because only test databases were involved, but it’s good to run a tidy shop so the problem had to be addressed.
These databases form part of a Database Availability Group (DAG). Normally when a content index fails and needs to be reseeded, you simply run the Update-MailboxDatabaseCopy cmdlet and specify the CatalogOnly switch to request Exchange to reseed the content index from a good copy belonging to another database copy. But when you’re running a single-copy database there’s no other good copy (of either the database or the content index) hanging around waiting to be called into play.
Hmmm… TechNet wasn’t too helpful on the topic of reseeding a single-copy database and the suggestions offered in various web sites all leaned toward a complete rebuild of the index. Eventually I decided to go with that plan because there didn’t seem to be any good alternative. Note that users are able to keep on working with their mailboxes even when a database has a failed index. It just means that searches performed with OWA will be slower.
In any case, I used the following steps to get my three errant databases back to good health.
- Stopped the Microsoft Exchange Search service
- Stopped the Microsoft Exchange Search Host Controller service
- Dismounted the database
- Deleted the [guid.single] folder in the folder holding the database file. Guid is the Globally Unique Identifier for the database. You can find this by running Get-MailboxDatabase database-name | Select Guid. For example, the folder you need would be something like d:\Databases\DB2\79c03cca-9b53-4959-982a-8773591c5f70.single
- Restart the Microsoft Exchange Search Host Controller service
- Restart the Microsoft Exchange Search service
- Remount the database
As each database was remounted, the Search service recognized that its content index was missing and began the process of rebuilding the index. The content index status will remain as “Failed” until the rebuild is complete. A couple of minutes later all was well and the server reported full health. Of course, this was a relatively small database so the Search Foundation didn’t have too much work to do to recreate the content index. The process will take longer as the database size grows; it is definitely not something that you will want to do if the index fails for a large database.
I’m not recommending that you delete folders on a production server. Then again, I hope that on a production server you’ll have more than a single copy of any database within the DAG (remember, two copies provides basic redundancy, three is much better, and four copies provides a warm blanket feeling) and will therefore be able to run Update-MailboxDatabaseCopy. But if you do get into a hole, you might be able to use the steps outlined above to get out of it. And that’s always a good thing, isn’t it?
Follow Tony @12Knocksinna