A recent question from a reader focuses on the need to determine the last logon time for Exchange users. The organization is in the middle of moving to Exchange Online and some management reporting scripts that are used do not deliver the same results when run against Exchange Online mailboxes.

A quick discussion revealed that the script uses the Get-MailboxStatistics cmdlet to retrieve login information, specifically the LastLoggedOnUserAccount property. For instance, if we use the cmdlet to look at an Exchange Online mailbox, you might see something like this:

[PS] C:\> Get-MailboxStatistics –Identity Ben.Owens@Office365ExchangeBook.com | Format-Table *Log*

LastLoggedOnUserAccount LastLogoffTime LastLogonTime

----------------------- -------------- -------------

                                       10-Mar-16 5:39:54 PM

The LastLogonTime property reports the last time the mailbox was logged onto by a user. Well, it does and it doesn’t. This property used to report the last access time by a user accurately, but that was when only users opened mailboxes. Today, Exchange Online has a bunch of mailbox assistants that access mailboxes on behalf of users to process items (for example, the Mailbox Folder Assistant), so you cannot depend on this time stamp anymore. All it tells you is the last time something opened the mailbox – and often that something is a background process.

The LastLoggedOnuserAccount property, which is used to report the name of the last account to connect to the mailbox is blank. In some instances, this information could be important because it’s not necessarily the case that the mailbox owner is the account that connects to the mailbox. Perhaps it was another account with full access permission.

Two problems are faced here. First, the ability of the Get-MailboxStatistics cmdlet to report the name of the account that logged on to a mailbox was deprecated in Exchange 2013. If you examine mailbox properties through the Exchange Administration Center (on-premises or Online), you’ll see that only the last logon time is reported.

Second, Office 365 operates a directory of record (Azure Active Directory) and workload-specific directories. The directory of record is the source of authority for account information while the workload directories hold information specific to a workload. Exchange Online does not have the same intimate relationship with the directory of record as exists on-premises where Active Directory is used for everything.

For example, Exchange Online obviously needs to hold information about mailboxes that are not necessarily required by every Office 365 account as some accounts don’t use Exchange. This information is held in EXODS (Exchange Online Directory Services). An EXODS instance is operated for each Exchange Online forest and multiple forests support the various Office 365 regions (U.S., EMEA, India, etc.). Similar arrangements exist for SharePoint Online (SPODS), Skype for Business (LYODS), and Yammer.

Synchronization routines are in place to keep Azure Active Directory and the workload directories aligned. Normally, synchronization is very fast and a change made to an attribute in a workload directory is updated in Azure Active Directory in a matter of seconds.

However, because Azure Active Directory is the directory of record, you have to retrieve information about user logons from it rather than using a cmdlet like Get-MailboxStatistics that operates exclusively against EXODS. Thus, we need to use the Search-UnifiedAuditLog cmdlet to interrogate the Azure Active Directory data held in the SIEM-like repository for audit information drawn from workloads across Office 365.

The unified auditing repository is the same data that is searched by the Office 365 Activity Report option in the Compliance Center. It is populated by feeds from multiple sources drawn from across the service, including Exchange mailbox and admin audit data. The idea is that the number of sources will be enhanced over time so that audit information from every Office 365 workload will be available through a common interface, which seems like a good thing.

Here’s an example to search the unified audit repository for logons by a specific user (you can pass several user identifiers separated by commas):

[PS] C:\> Search-UnifiedAuditLog -Operations PasswordLogonInitialAuthUsingPassword -StartDate 9-Mar-2016 -EndDate 10-Mar-2016 –UserIds Ben.Owens@Office365ExchangeBook.com | Format-Table UserIds, Operations, CreationDate

UserIds                             Operations                            CreationDate

-------                             ----------                            ------------

Ben.Owens@Office365ExchangeBook.com PasswordLogonInitialAuthUsingPassword 10-Mar-16 5:41:06 PM

Depending on when the data is imported from a feed, it can take up to 12 hours before data from a source appears in the repository. In some cases, you can get the data that you need sooner by executing searches against Exchange admin audit log or mailbox audit log data. However, the caveat here is that sometimes those searches don’t work as well as they should.

Audit events return a property called AuditData in JSON format. You can look at the raw data and pick things out but it’s often easier to use the ConvertFrom-JSON cmdlet to interpret the data. First capture the audit event you want to examine into a variable and then run it through the cmdlet. For example, this code finds some records, stuffs it into the $Audit variable, and uses the cmdlet to view the audit data of the sixth record in the set.

[PS] C:\> $Audit = Search-UnifiedAuditLog -Operations PasswordLogonInitialAuthUsingPassword -StartDate 9-Mar-2016 -EndDate 10-Mar-2016 –UserIds Ben.Owens@Office365ExchangeBook.com | Format-Table UserIds, Operations, CreationDate

[PS] C:\> ConvertFrom-JSon $Audit.AuditData[5]

The client field gives us a clue as to what application was involved. For instance, if you see “Outlook” here you know that an Outlook client has authenticated against the account. If “Exchange” is shown, it means that an ActiveSync client connected. However, these events record instances when clients were forced to go through the process of entering a password. If cached credentials are used, they won’t be recorded. We need to focus on audit events for a different operation. The MailboxLogin operation seems like a good choice:

[PS] C:\> Search-UnifiedAuditLog -Operations MailboxLogin -StartDate 9-Mar-2016 -EndDate 10-Mar-2016 –UserIds Tony.Redmond@Redmondassociates.org | Format-Table UserIds, Operations, CreationDate

By now you’ve realized the flaw in this explanation. No connection exists between the audit data and the mailbox logon data. We know that some account connected to a mailbox at a certain time. We can surmise that it might be the mailbox owner by checking the logon data from Azure Active Directory, especially if the user logged on to their account just before the connection was made to the mailbox. But we can’t prove that the mailbox owner was the account that connected to the mailbox.

All of this proves that you can’t take it for granted that scripts or other techniques that work well on-premises will transition flawlessly into the cloud. Office 365 is a completely different environment (in so many different ways) than any on-premises environment. Some work is probably required to review and, if necessary, to update your favorite scripts before they’ll work as well in the cloud.

The good news is that Microsoft’s approach to build a unified auditing repository for Office 365 encompassing all workloads is commendable and is likely to improve in terms of scope and capability over time. Isn’t progress wonderful?

Follow Tony @12Knocksinna