Reflecting the large number of iPhones and iPads that are connected to Exchange mailboxes via ActiveSync, I received a lot of email after I posted a note on February 7 reporting the problems with excessive transaction log generation that Exchange servers experience after iOS devices are upgraded to iOS 6.1. Microsoft’s formal knowledge base article (KB2814847) on the topic is also available for your reading pleasure. Microsoft offers three ways to resolve the problem, including the creation of a custom throttling policy, which I do not cover here.
Ever since the problem appeared, people have been asking what to do to control these pesky iOS 6.1 devices. The problem can be a delicate matter because senior management are often fans of Apple devices and it wouldn’t go down well if IT suddenly imposed a blanket ban on iOS 6.1. The stress on Exchange would certainly ease if the rate of transaction log generation reduces but the stress level of iPhone users might increase if they were to receive a message from Exchange to tell them that their device is blocked.
Apparently Apple is responding by rushing a new build (6.1.1) out to address the issues noted with Exchange as well as some other problems with 3G performance and degraded battery life. The new version is being directed to iPhone 4S devices first. If you don’t want to wait for the new build to get to your iOS devices and are interested in the details of how to use ActiveSync device access rules to control access to iOS 6.1 (and other devices), fellow MVP Paul Cunningham has written up the necessary steps to put the blocks in place. A big health warning applies before any block is imposed: test, test, and test again.
Some correspondents have found that they can handle the problem by deleting the device partnership and forcing a complete resynchronization and indeed, this is one of the suggestions listed in KB2814847. It is reasonable to assume that this step causes the iOS mail app to use different code to synchronize the entire mailbox rather than just the meeting requests that seem to lie at the root of the problem.
A device partnership essentially links a mailbox and a mobile device and allows Exchange to track what happens with the device. You can see details of a partnership using the Get-ActiveSyncDevice command (works with Exchange 2010 and 2013) and you can remove a partnership with the Remove-ActiveSyncDevice command. Two small problems get in the way. First, you have to pass the identity of the partnership that you want to remove. Second, many users are associated with multiple partnerships because they use (or have used) multiple devices to synchronize with Exchange via ActiveSync.
Of course, forcing every single iOS device to resynchronize is neither good for user blood pressure nor your server, so it’s good to know what mailbox is linked to the problem. One good way of doing this is to run the Exchange User Monitor (ExMon) utility to monitor the demand that mailboxes are making on a server. The problem mailboxes will be easily seen as they are the ones that consume far more resources than any other mailbox. Another good suggestion is to use Microsoft’s Log Parser studio (this post provides good guidelines for what you need to do) to extract information from the IIS logs to focus on the misbehaving clients. Others have found that the ActiveSync Report script posted on EHLO is useful.
Whatever method you use to identify the mailboxes, take a note of their names and then run the Get-ActiveSyncDeviceStatistics command to confirm that they are using iOS 6.1 devices. If they are not, the excessive mailbox activity can only be accounted by their insertion of new set of batteries recently. For example, this command returns a list of the ActiveSync partnerships that are known for my mailbox:
Get-ActiveSyncDeviceStatistics –Mailbox Tredmond | Format-Table DeviceID, DeviceType, DeviceOS
DeviceOS will start with “IOS 6.1” for any device running iOS 6.1. DeviceID contains the identifier for the partnership that we need to remove, so an amended command will remove the partnership. Get-ActiveSyncDeviceStatistics returns all known partnerships for the mailbox, so the filter is absolutely needed to remove just the partnerships associated with iOS 6.1:
Get-ActiveSyncDeviceStatistics –Mailbox TRedmond | ? {$_.DeviceOS –match “iOS 6.1”} | Remove-ActiveSyncDevice
A polite phone call to the user might be in order to avoid the frantic help desk call that will ensue when they realize that their device needs to resynchronize.
Microsoft is also struggling with the same operational concerns over iOS 6.1 device with their Exchange Online service and have implemented throttles to ensure that problematic devices are blocked. Seems like the right approach to take – never let badly functioning devices (for hardware or software reasons) compromise the overall stability of a service.
Follow Tony @12Knocksinna
Thoughtsofanidlemind's Blog 
Tony, do you see any issues with turning on circular logging on the problematic databases as a short term fix until the update is released?
Thanks.
No, providing that the database is protected by multiple copies within a DAG. Running a production database with circular logging enabled and no copies available sounds like a tight-wire act to me. I’m not fond of walking thin wires at height.
TR
Circular logging would be a good solution if this problem were happening with every iOS user consistently. Unfortunately I’ve only seen this issue once – and don’t even have enough facts to convince me the low growth was caused by iOS – so I’m not willing to risk my EDB restore options for what may never happen again.
We have seen it on two database and it tends to generate about 1 GB of transaction logs every 10 minutes or so. There are definitely multiple copies of the databases, and we are also backing up the databases on schedule, as the backup application we are using is circular logging aware. I just don’t want to have a database dismount in the middle of the night while sleeping. A dismounted DB is not something you would want to deal with during your morning cup of coffee.
The log growth is coming from iOS devices. You can see it in the IIS logs on the CAS boxes- excessive MeetingResponse numbers that are in the tens of thousands.
Given that you have multiple copies of the database, why are you not using circular logging anyway? In any case, enabling circular logging won’t hurt unless it compromises the backup that you use in some way (something to check…)
TR
Pingback: iOS 6.1 causing very high log generation on Exchange | Jason (Izzy) Sherry's Blog
Thankfully I’m running a multi-copy DAG with Circular logging. So far Exchange hasn’t batted an eyelid. I just checked with Get-ActiveSyncDeviceStatistics and we only have ~200 6.1 devices so far though
Pingback: How to Block iOS 6.1 ActiveSync Devices from Exchange Server 2010
Pingback: msdigest.net » Apple iOS 6.1 causing excessive transaction log growth on Exchange 2010
We have nearly 1800 iOS6.1devices out of a 4500 iOS ActiveSync community. So far, using the Log Parser Studio technique only 2 end users seem to be exhibiting the symptoms and then it seems to be a low number of hits reported. Perhaps they don’t have Push enabled? We haven’t noticed a massive difference in transaction logging yet. My worry is the problem amplifying when we aren’t around, but the impact and reputational damage of blocking 1800 mobile devices is a concern. The most disappointing thing is Apple not acknowledging the problem and at least to get an idea that they are working on a solution. I’ve not heard from anywhere that iOS6.1.1 will solve our issue, but is Apple’s fix for the significant impact they caused the Vodafone network in the UK.
BYOD can be such a challenge!
Hi John,
You might just be lucky and your iOS 6.1 users aren’t dealing with calendar entries in such a way that the bug that causes excessive growth of transaction logs is exposed. Stay lucky!
In the meantime, keep an eye on those logs too…
TR
Pingback: EHLO! | iOS 6.1 based device
Tony, to your “how MSFT is tackling this issue in Office 365 Exchange Online space”‘ concern, as mentioned in this KB article: http://support.microsoft.com/kb/2748176, “Exchange Online has implemented EAS throttling to manage and maintain optimal performance of the Exchange Online environment in Office 365.” & “When a mobile device acts in a manner that can adversely affect Exchange Online service performance, the device is put into an Access Denied state for a short time. For example, this occurs if a device sends too many identical Sync commands to the service for a particular folder in a very short period.” The same feature-set is now available in Exchange 2013 for on-premises customers. More details are here: http://technet.microsoft.com/en-us/library/jj218622.aspx
Thanks for the update. Microsoft published what they were up to after I posted the blog. It’s reasonable what they have done to protect Office 365.
TR
Apple have now released a support article:
http://support.apple.com/kb/TS4532
At least they’ve now acknowledged the issue and identified a fix. Hopefully, it’ll be very soon, but I have my doubts.
It is also possible to delete the device partnership through the Exchange Control Panel (ECP). This is a user friendly interface to obtain the OS level and easily remove the partnership.
Absolutely. However, EMS will do a better job when there are more than one or two devices to track down…
Has anyone done the quick “Calendar Toggle” steps by Apple, and have they confirmed it fixed the transaction log growth & CPU/Memory usage issue? Thanks in advance.
Hello,
We are planning to create a new throttling policy to limit the impact on the server. I am not sure if it will control log growth, but it should help with potential high CPU usage on the CAS boxes. Hoping will help with the log growth too.
Policy below:
New-ThrottlingPolicy -Name iOS6.1 -EASPercentTimeInCAS 10 -EASPercentTimeInMailboxRPC 10 -EASPercentTimeInAD 10
I am not endorsing this throttling policy, but we plan on creating and running it on certain iPhone users. We will add users to the policy if they are showing up on our list for users with an unusally high rate of MeetingResponse numbers.
I have not tried the calendar toggle, but thought a hard reboot of the phone would sever those requests and a hard reboot of a iOS 6.1 iPhone did not help at all.
I am sure there are many shops out there really feeling the impact. Especially if they did not allow for transaction log growth and did not have monitoring on the transaction log volumes.
I just wish Apple would get their act together. Do they not have an Exchange test environment??
We ran into a pesky and unexpected side effect related to this bug today where cancelled recurring meeting occurrences flooded the dumpster of a few users impacting the free space of some databases.
Pingback: EHLO! | iOS 6.1.2 Fixes an Exchange calendar bug