Like every version since Exchange 2000, Exchange 2010 has a huge dependency on Active Directory. Essentially, if you don’t deploy and manage Active Directory in the right way, you have little or no chance of being successful with an Exchange 2010 deployment. If you disbelieve me, simply break out the schema update files from the Exchange 2010 installation kit to see the number of changes that Exchange makes to the Active Directory to store all the information it needs about mail-enabled objects, its configuration, and other objects.
Greenfield deployments have always been the easiest for Exchange. Things become more interesting when you have to cope with the debris of previous deployments. Of course, the problems are entirely due to other administrators who are now long gone and have left you to clean up the mess. Some of those issues might be lingering problems in Active Directory that come to the surface during the deployment of Exchange 2010 and that’s what I want to cover here.
Recently, online forums have described two problems that have caused administrators grief as they work with Exchange 2010. Both share the same fundamental root cause in that Exchange 2010 is unable, for some reason, to update or create information in Active Directory. The first issue I’ll discuss is when you can’t create a new mailbox move request for some reason. In this example, we create a new move request to move my mailbox by running the New-MoveRequest cmdlet in the Exchange Management Shell (EMS):
PS C:> New-MoveRequest -Identity 'Redmond' -BadItemLimit 10 -MRSServer exserver1.contoso.com -TargetDatabase DB2
New-MoveRequest : Active Directory operation failed on AD-Root.contoso.com. This error is not retriable. Additional information: Insufficient access rights
to perform the operation.
Active directory response: 00002098: SecErr: DSID-03150BB9, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0
At line:1 char:16
+ New-MoveRequest <<<< Redmond
+ CategoryInfo : NotSpecified: (0:Int32) [New-MoveRequest], ADOperationException
+ FullyQualifiedErrorId : A7516A9,Microsoft.Exchange.Management.RecipientTasks.NewMoveRequest
Bang! The move request fails immediately and we’re told that the problem is with Active Directory. We can also see that it’s due to insufficient access rights. The reason here is that a move request causes the Mailbox Replication Service (MRS) to update several attributes in the user object that MRS uses to track and report the progress of a move. Some of the attributes are static and some (like the move status) are updated as the move progresses. You can retrieve this data with the Get-MoveRequest cmdlet. The attributes are:
- msExchMailboxMoveBatchName (batch name for the move request, if specified)
- msExchMailboxMoveFlags (any flags specified for the move)
- msExchMailboxMoveRemoteHostName (the name of the remote host if MRS is pushing mailbox data to a legacy Exchange server)
- msExchMailboxMoveSourceMDBLink (the source database)
- msExchMailboxMoveStatus (the current status – for example, “Queued”)
- msExchMailboxMoveTargetMDBLink (the target database)
These attributes are removed from the user object when the move request is cleared with the Remove-MoveRequest cmdlet (or from EMC). However, to begin a move, MRS has to be able to update Active Directory and if it can’t, the move hits a brick wall and fails immediately. There is no workaround and in Exchange 2010 RTM there’s also a nasty side-effect of the problem in that MRS creates an orphaned move request for the mailbox in the system mailbox of the source database. If the administrator fixes the Active Directory permissions problem and attempt to run New-MoveRequest again, MRS detects the orphaned move request in the system mailbox and will refuse to create the move request. You then have to remove the orphaned move request by running the Remove-MoveRequest cmdlet and specifying the source database name in the –MoveRequestQueue parameter. For example:
Remove-MoveRequest -MoveRequestQueue 'DB2'
Microsoft improved the situation in Exchange 2010 SP1 by removing the need to clean up the orphaned move request. In SP1, if you create a move request for a mailbox where an orphaned move request exists, MRS overwrites the orphan move request with the information for the new move request.
But why would such a problem occur? After all, are Exchange 2010 system components not granted access to Active Directory through the Exchange Trusted Subsystem? The answer is that the Exchange Trusted Subsystem indeed has very privileged and powerful access to Active Directory but it cannot deal with situations where the Access Control Entries (ACEs) that Exchange depends upon are not stamped onto user objects for some reason. When this happens, any attempt by Exchange (through MRS or another component) to update the object is declined by Active Directory and we get into the situation of the failed move request.
Another example of where Exchange 2010 runs into permission problems is when it attempts to create child objects for devices that synchronize with Exchange using ActiveSync. The objects are msExchangeActiveSyncDevices (the list of devices that a user mailbox has used for synchronization) and msExchangeActiveSyncDevice (for each device to track its synchronization characteristics). The first time a mailbox attempts to synchronize using ActiveSync, Exchange 2010 attempts to create these child objects for the user object. If this succeeds, synchronization proceeds and all is well. If Exchange is blocked by Active Directory permissions, synchronization fails and event 1053 for MSExchange ActiveSync is logged in the Application Event Log. The event detail is similar to the MRS error described above – unable to create the container under the user object.
AdminSDHolder is one of the most common reasons why ACEs don’t get stamped on objects. The logic behind AdminSDHolder is impeccable as it helps to secure user accounts that possess elevated Windows permissions. Administrator accounts – ones that are members of protected groups such as “Administrators” are protected by AdminSDHolder and have ACL inheritance turned off. A process called SDPROP runs every 60 minutes (default interval) on the PDC to check the ACL of protected groups such as Administrators and reset their inherited permissions to that specified for the AdminSDHolder object. The account shown in the screen below is an example – note the circled checkbox – and the really interesting thing is that this can happen as the result of group membership that has long since been removed from the user account. The best way to check for accounts that are under the control of AdminSDHolder is to search for accounts whose admincount attribute is set to 1 (one). Do this as follows:
1. Start a Windows PowerShell session
2. Import the Active Directory module
Import-Module ActiveDirectory
3. Execute the Get-ADUser cmdlet to find the user accounts with admincount = 1
Get-ADUser -LDAPFilter "(objectcategory=person)(samaccountname=*)(admincount=1)"
User account with inherited permissions turned off
AdminSDHolder is the most common cause but any action that stops the ACEs necessary to allow Exchange to manipulate user objects will cause a lot of problems for Exchange 2010.
To fix the problem for a user object, you have to reset its admincount attribute to 0 (zero) (I use ADSIEdit for this purpose) and then check the “Include inheritable permissions from this object’s parent” box. If you don’t reset admincount, you will find that everything works OK until the next time that SDPROP runs, at which time the checkbox will be unset again.
It’s a bad, bad, bad idea to mail-enable administrative accounts as there should be a clear separation between accounts that are used to manage servers and those that people use for access to applications such as email. This is as true today as it was in the days of ALL-IN-1 and IBM PROFS as it prevents accidents happening when administrators perform tasks with their elevated permissions turned on – like deleting files that they really should have kept. It’s always best to keep administrative and “other work” activities separated with different accounts, even if it is a pain to switch accounts sometimes just to read some email – but at least your mobile devices will keep working.
The word is that the problems described here tend to affect highly privileged user objects that were created before the deployment of Exchange 2010. Objects created subsequently don’t seem to have the same problems. I can’t quite work out why the newer objects are more resistant but I can report that I have had no problems moving mailboxes for users who don’t have inherited permissions using Exchange 2010 SP1.
After all of that, the bottom line therefore is that migration projects that are moving from Exchange 2003 or Exchange 2007 to Exchange 2010 may run into problems moving or synchronizing mailboxes if they have accounts with elevated administrative permissions that are used to access email.
– Tony
Follow Tony @12Knocksinna
For more information about Exchange 2010 SP1, see Microsoft Exchange Server 2010 Inside Out
Thoughtsofanidlemind's Blog 
Tony, off topic question here but I have not seen anyone speak of the cross-site connection ability that was supposed to be in SP1. It was mentioned in TechEd. Have you heard anything?
Microsoft had to cut the cross-site connection control feature very late on during SP1 development. I believe that the feature should reappear soon when Microsoft has had a chance to do more development and testing to make sure that everything works as it should in different circumstances.
TR
hello, excuse my ignorance, as I open the window “Advanced Security Settings for …”? And then what steps should I do to fix it?
I also have a problem of access to ActiveSync for a user.
Thanks
Hi Leonardo,
Did you follow the advice in the blog?
>>>>>
To fix the problem for a user object, you have to reset its admincount attribute to 0 (zero) (I use ADSIEdit for this purpose) and then check the “Include inheritable permissions from this object’s parent” box. If you don’t reset admincount, you will find that everything works OK until the next time that SDPROP runs, at which time the checkbox will be unset again.
Thanks Tony..You have been most helpful. I could not create a particular user mailbox in Exchange 2010. Following your suggestion helped resolve it. Thanks.
Call me blonde
I have the same problem you are having. I cant edit the cirtificate store on Active Directory, I can’t create a new mailbox on the console. (all the fields a locked when i try to edit an existing user in the console)
iIhave read through the blogpost and i am unable to find the solution.
Can someone please help….
Have you called Microsoft support? This might be the quickest way of resolving the issue as it can be awfully frustrating and sometimes misleading to hunt through web sites looking for a solution that matches your problem.
Are you running with Admin permissions using the account used to install Exchange. If not, try that.
TR
Thanks very much Tony. I had this exact problem on ActiveSync and your post helped solve my ActiveSync device issues. Thanks again!
Hi Tony.
We have a bizarre issue where some of the users of the domain have the admincount set to 1. Doesn’t affect everyone and they don’t seem to be in any protected groups. This has happened at a couple of our clients when we migrated from 2k7 to 2k10. There are far too many user to go through manually and change the attribute manually. Do you know of a script that could change the attribute on a whole OU?
Thanks.
Hi Jay,
I don’t have a script to hand but I bet that you could find out how to build one by consulting something like Brian Desmond’s Active Directory book. It should be simple enough.
TR
I have just completed our Exchange upgrade over the last 4 days and hit most/all of these problem. I found these fixes quiet quickly (with a little help from Mr Google) but now the dust has settled I wanted to try and understand why I had to do what I did. This documentation helped me clear things up immediately and being the only IT personnel at my company really helped when I have no one to bounce off!
Great article!
I wonder if the issues I have is related to something mentioned in this article though they are a little different. I have some intermittent problems connecting to autodiscover and prompted to logon (even though Im on the LAN and Integrated security should be used). Llike going to https://MyClientAccessArray.mydomain.com/autodiscover/autodiscover.xml causes logon prompt which fails if I try to enter correct credentials. If at this point I refresh the browser sometimes (?) Im in. Checked everything and it looks as should be according to your book. Is there anything special I need to set to make it work properly?
Have you tried running Outlook in debug mode to see what’s captured in the log? It might provide a pointer to where the problem lies…
TR
Has anyone run into ADMT migration issues due to the msexchangeactivesyncdevices being leaf objects under a user. We are currently facing that issue and is in the ADMT documentation that users cannot have a leaf
Good Article …Thanks Tony…
Thanks. this was a problem for one of my users.
Thanks for this, I had the exact problem and this has fixed it.
Also enjoying your “Inside Out” Exchange book which is xlnt.
Glad you like the blog.
1. Write about something you’re passionate about.
2. Write something different – don’t duplicate what else is available on the web.
3. Add value, not words.
If you manage all three, your blog will be good.
TR
Very helpful. Thanks for the information. Saved me a good amount of time troubleshooting.
Tony,
What about accounts that for physical security (people have threatened to kill employees in that position, and in fact, I understand some have carried out that threat) CANNOT inherit the permission in AD. These are normal users accounts. What rights do I need to delegate to Exchange to get that to work around the blocked inheritance.
Daniel,
I don’t really understand the question. Why would someone be under threat if they were asked to use a normal user account to access Exchange? Sounds very strange to me…
TR
Thanks for the quick response: Here are the answers to the different ways I see you could be confused.
Why can’t inheritance happen?
It’s the ability of someone to find where they work, come in, and shoot them. The accounts in this OU have inheritance broken so that information is not available to anyone with a domain account (by using ldap tools to read things like “office” from the user account).
What do you need?
I need to know which rights the account Exchange needs (I think it’s Exchange Trusted Subsystem) to have reapplied to the OU where inheritance is broken to allow Exchange to function. We ran into the same issue with our AD Domain Admin accounts, but that was resolved by moving our accounts that get email out of Admin use, using just our personal Admin Accounts. That is not an option for these accounts.
Thanks again!
Danny
OK. You’re definitely an edge case that the Exchange developers probably didn’t take into account when they designed their solution. You need to talk to Microsoft about this – or ask one of the deep Active Directory guys to help – Brian Desmond (briandesmond.com) would be a good choice. Coming up with a solution that meets your needs and doesn’t break something will require work and a level of expertise and acquaintance with Active Directory that I don’t have.
TR
I wrote up a powershell script to help me deal with undoing a mistake that threw me into this situation. I hope someone will find it usefull:
#Accidental SD Prop Fixer
#Wrote by Mujizac
#Resets admincount to null. Sets the user object to include inheritable permissions.
#Created because I accidently added all my users to printer operators group which jacked up thier #user objects
Import-Module ActiveDirectory
#Be sure to edit the target ou to fit your domain and target.
$targetou=”OU=Users,DC=domain,DC=local”
get-aduser -ldapfilter “(objectcategory=person)(samaccountname=*)(admincount=1)” -SearchScope Subtree -SearchBase $targetou | foreach-object{
Set-AdObject -identity $_ -clear admincount
$dn = $_.DistinguishedName
$user = [ADSI]”LDAP://$dn”
$acl = $user.objectSecurity
write-host “######################”
write-host $dn
write-host “Current Setting of Access Rules Protection:”
write-host $acl.AreAccessRulesProtected
$isProtected = $false
$preserveinheritance = $true
$acl.SetAccessRuleProtection($isProtected,$preserveinheritance)
$inherited = $acl.AreAccessRulesProtected
$user.commitchanges()
write-host “Changed to:”
write-host $acl.AreAccessRulesProtected
}
Hi Zac,
Thanks for sharing your script.
TR
I finally have the answer for this! And it is as easy as pie. I actually worked with MS on this.
1. Go to ADUC
2. Expand the System OU
3. Go into the properties of the AdminSDHolder
4. Choose the Security tab
5. Highlight Exchange Servers (domain\Exchange Servers)
6. Check the box Full Control
7. Click OK to go back to the main screen
8. Wait for SDProp to run (usually takes about an hour)
9. Voila! You are done. The ActiveSync for any user in a protected group will start working and any future user you move over to 2010 will automatically work with no intervention. The only time they wont work is during the move.
As far as this being a security risk – your network is only as secure as you make it. If you have good firewalls in place and follow best practices, you should be alright.
Pingback: Exchange 2010 move mailboxes error Access Denied | swissiws
I created a sevice account to move the mailboxes from E2K7, E2K10 servers to E2K13 servers. I made this service account member of Organization Management group. I am also using this account to create new mailbox in Exchange 2013. Whenever I am trying to create the new mailbox by using the service account the below error is showing:
This error is not retriable. Additional information: Insufficient access rights to perform the operation. Active directory response: 00002098: SecErr: DSID-03150E49, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0
I able to create mailbox of the same user account which I tried on Exchange 2013 in Exchange 2007. I am also able to move the mailbox after creating it. I checked the inheritance on all the containers in the configuration partition and its there. Also Exchange trusted subsystem and Organization Management group are present with inheritance enabled on all levels.
Please suggest what could be other possible steps and reason for this weired permissions behavior.
One more thing to add is that our Exchange Servers are on Windows 2012 R2 and Exchange 2007/2010 are on Windows 2008 R2.
I suggest that you log a call with Microsoft Support to track down and solve the root cause for your problem. This is not a one-stop guaranteed solution centre for Exchange and there’s no way that I or anyone else can tell what;s going on without running some tests on your systems. Anyway, free support is worth exactly that much.
TR
Hi everyone,
I had the same problem that you had “Insufficient access rights”.
Every time I tried to create a user mailbox, group or granting Send As permission to a user I got this errors, then I solved my problem giving permission to Exchange Trusted Subsystem group to all my AD users.
Exchange servers through the Exchange Trusted Subsystem group have permissions to create, modify or delete security principals in Active Directory.
Then you need to check if the following things match:
1- Check if Exchange Trusted Subsystem is a member of the Exchange Windows Permissions (it allow Exchange to create AD DS Objects).
2- Grant permission to Exchange Trusted Subsystem group in the AD object which you want to change, for example:
Go to Active Directory users and computers, select an user or a OU, click right bottom-properties-
>security tab and grant full control permission to Exchange Trusted Subsystem Group. The best to do is to apply that to a entire OU and applies to this object and all descendant objects.
All users, OU or any objects that Exchange Servers needs to modify, have to have permission to Exchange Trusted Subsystem Group.
I really encourage you take a look at AdminSDHolder, which is well explained here in this blog.
We took ownership of what looks like a half-baked Exchange 2013 deployment but giving Exchange Trusted Subsystem Full control permissions on the user within ADUC is what helped us. We are currently doing it for every user that we need to move/modify. Sort of unease with making the change at the OU or domain level.
I’m not so sure that I would attempt to use the permissions gained through the Exchange Trusted Subsystem for this… but details are sparse and it’s hard to comment. I would log a call with Microsoft to make sure that they have a chance to find out why things are so royally messed up.
Same issue with a new installation here. And grating Full Access to Exchange Trusted Subsystem for a particular user or OU seems to do the trick.
However – for new objects such as mailboxes for instance, i am wondering how to grant this access for the Exchange Trusted Subsystem as the object does not exist? Runnig the ” new-mailbox” from the Shell throws the exact same error, and the “Edit” button in EAC is greyed out.
Was this an issue for any of you , and did it get solved by one of the suggestions?