The Store Driver is a very important Exchange component. Running on all hub transport servers, its function is to provide the mechanism to deliver inbound messages to mailbox databases. Unlike Exchange 2003 and previous versions, all messages go through a hub transport server, even if they are sent between two mailboxes in the same mailbox database.
Exchange 2010 SP1 includes new code to throttle or manage connections from the Store Driver to mailbox servers as it delivers messages in such a way that Exchange can isolate and limit the effect of any faults that occur. Most of the time, you’ll be unaware that throttling applies to the Store Driver. However, if you receive non-delivery notification (NDR) messages that report problems from “STOREDRV” (the Store Driver), your Exchange hub transport servers might be hitting a limit. For example, an entry in Microsoft’s Exchange 2010 forum reports this error:
432 4.3.2 STOREDRV.Deliver; recipient thread limit exceeded
As a quick search for “STOREDRV 432” reveals, the Store Driver issues a 432 NDR when it encounters other problems, so the important thing here is the text reporting that the “recipient thread limit exceeded”.
In most cases, the problem seems to be encountered with mailboxes that receive very heavy traffic such as those used for journaling. As you probably know, Exchange allows you to capture journal reports to any valid SMTP address. This facility allows you to direct journal traffic to a third-party journaling application but if you send journal reports to an Exchange mailbox, perhaps as an interim step before the reports are later moved to an archiving application, it’s possible that Exchange will encounter a throttle limit as the Store Driver attempts to deliver messages concurrently at times of peak demand. By default, Exchange 2010 SP1 will only attempt a single concurrent delivery to a mailbox to ensure that all of the mailboxes in a database receive a similar quality of service. This is what you’d want, but you can see that a journal mailbox might have many messages directed towards it concurrently.
The solution is to update the EdgeTransport.exe.config XML configuration file on the hub transport server to add two new keys to force Exchange to allow more than one concurrent delivery to a mailbox. Any text editor can be used to update the configuration file but it is best to update a test server first and measure the effectiveness of the change before you introduce it into production.
It is also best to make the update to all hub transport servers in the site that supports the journal mailbox so that the Store Driver has the same behaviour on all servers.
The new keys and their values are:
<add key="RecipientThreadLimit" value="2" />
<add key="MaxMailboxDeliveryPerMdbConnections" value="3" />
You should restart the Exchange Transport service after making the change to force the Store Driver to respect the new setting.
As documented on Page 397 in chapter 7 of Microsoft Exchange Server 2010 Inside Out, also available at Amazon.co.uk, the throttling parameters that are respected by the Store Driver in Exchange 2010 SP1 are as shown in the following table.
|Hub transport server to mailbox server||Concurrent delivery to a mailbox: Only one message can be delivered concurrently to the same mailbox. This limit is in place to ensure that all the mailboxes in a database receive a similar speed and quality of service.|
|Concurrent delivery to a database: Only two messages can be delivered concurrently to the same mailbox database. This limit prevents a problem with one database affecting all the other mailbox databases on a server. The problem may be caused by an underlying hardware issue such as a failing disk or storage controller or a software bug. Exchange may reduce this limit automatically if it senses that the performance of a mailbox database is lower than normal.|
|Concurrent delivery to a mailbox server: Only twenty-four messages can be delivered concurrently to the same mailbox server. A hub transport server can deliver to any mailbox server in the site. This limit prevents a problem with one mailbox server (that might be experiencing heavy load) from affecting delivery to all other mailbox servers in the site.|
|Concurrent delivery per message across all mailbox servers: Only twelve concurrent deliveries per individual message can be attempted to all mailbox servers in the site. This limit prevents a problem with a corrupt or expensive message (such as one with a number of large attachments or several hundred recipients in the message header) from absorbing all available connections and therefore stopping or slowing down delivery from a hub transport server to the mailbox server within a site.|
|Mailbox server to hub transport server||Concurrent submission to a hub transport server: The limit for the maximum number of connections is calculated as five times the number of available mailbox processor cores. This limit controls how many concurrent submissions can be made from a mailbox server to all hub transport servers in the site.|
|Concurrent submission from a mailbox database: Only four connections can be performed concurrently from a mailbox database|
|Concurrent submissions from a single mailbox server: A hub transport server will only process at most twelve concurrent submissions from any single mailbox server.|
|`||Concurrent submissions to a hub transport server: A mailbox server will only make a maximum of fifteen concurrent submissions through any single hub transport server.|
|Site-wide hub transport to mailbox server||The upper limit for the number of connections from hub transport servers to mailbox servers across the site is 20 x the number of processor cores. For example, if there are two hub transport servers in the site, each equipped with four processor cores, the limit is 160 connections.|
|Site-wide mailbox servers to hub transport servers||The upper limit for the number of connections from mailbox servers to hub transport servers across the site is five times the number of processor cores. For example, if there are six mailbox servers in the site, each equipped with four processor cores, the limit is 100 connections.|
Connection limits imposed by Exchange on the Store Driver