Announcing a Kindle version of “Office 365 for Exchange Professionals”


After a certain amount of struggle, mostly associated with the need to provide book files formatted in a certain manner, the “Office 365 for Exchange Professionals” team is happy to announce that we now have a Kindle version of the book available on Amazon.com.

Our original intention was not to create a Kindle version. The work necessary to format a large and complex book (many tables, graphics, and footnotes over the 630-odd pages) didn’t seem worth the effort, especially when we had a perfectly good EPUB version already available. In particular, we weren’t happy with the way that code examples are treated. And the way that Amazon publishes Kindle books through its Kindle Direct Publishing (KDP) platform didn’t seem to match our desire to create frequent updates for the book.

However, we continued to receive a number of requests to support Kindle and so resolved to attack the problem again. After working through some “interesting” conversions, a Kindle edition is now available in Amazon stores worldwide.

We will continue to sell the book on ExchangeServerPro.com, where you can download PDF and EPUB versions. Amazon is easier for those who only want to read the book on a Kindle and like the way that Amazon wirelessly delivers content to Kindle devices. We actually believe that PDF on a PC is the best reading experience, but we want to support choice.

As mentioned above, we intend to issue frequent updates. The next edition should be available in September 2015 to coincide with the IT/DEV Connections conference in Las Vegas when all of the author team will be speaking at the event. When a new edition is available, we will release first on ExchangeServerPro.com and then work on the Kindle version. Once the new Kindle version is ready, we will publish it and withdraw the current edition from sale. The versions will be clearly marked as “May 2015 edition”, “September 2015 edition”, and so on, and we will include a description of the changes that are present in each version.

Right now, we are busy preparing the September 2015 edition. Many updates and new material have been incorporated in a number of chapters (35 additional pages to date) based on recent developments inside Office 365. More information will come as we have the chance to use some of the new technology that Microsoft announced at the recent Ignite conference, assuming that technology is available to Office 365 tenants by the start of September

Based on our experience to date, it seems like three-times-a-year might be a good cadence to attain for updates. Of course, that depends on having sufficient material to warrant an update, but signs are that Microsoft will continue to pump out changes into “the service” and those changes need to be examined, analyzed, and documented. That’s the task we have taken on and intend to see through. Hopefully you’ll join us on the journey.

Follow Tony @12Knocksinna

Posted in Cloud, Office 365 | Tagged , , , , | 1 Comment

Exchange Unwashed Digest – May 2015


May was the month of the Microsoft Ignite conference in Chicago. I’ve already provided my impressions of that event in another post, so I won’t rehash the topic here. There’s no doubt a lot of content was delivered at Ignite (I am still listening to recorded sessions) and so it’s natural that Ignite was a big influence over what appeared in my Exchange Unwashed blog on WindowsITPro.com during May 2015. Here’s what happened.

ESEUTIL is now the evil utility (May 28): Once an essential part of every Exchange administrator’s toolkit, the days of ESEUTIL now appear numbered. Microsoft has changed its support policy to positively discriminate against the use of ESEUTIL (there’s a lot of goodness in that decision), so why is ESEUTIL so bad all of a sudden. Well, there are good reasons…

Microsoft claims 35% of Exchange installed base is now on Office 365 (May 26): For the first time (that I know of) a Microsoft representative came out and said how much of the Exchange installed base they believed had gone over to the service. I don’t think it is 35%, but they do…

Updates make Office 365 Groups more useful (May 21): A number of long-awaited updates have appeared to make Office 365 Groups more useful. My particular favorite is the PowerShell support, which is now adequate. The updates to the document libraries are quite good too!

New engineering philosophies drive innovation within Office 365 (May 19): The old way of engineering products was to focus on just that product. Integration with other products happened almost as a result of blessed serendipity. Things are changing in the world of Office 365 as products become contributions to a software parts bin and new applications are built with the full spectrum of the service in mind. It’s a whole new way of doing business.

Eradicating EV stubs from Exchange mailboxes isn’t easy (May 14): An article published earlier in May discussed how Microsoft is aiming at Symantec Enterprise Vault with the new Office 365 import service. Well, there’s just one thing to spoil the party, and that’s how to get rid of all the stubs that are left behind in user mailboxes if you remove Enterprise Vault. Some third party products will do it for you. Others won’t. Or you can just annoy users with defunct stubs.

Why we shouldn’t care that Exchange 2016 really is Exchange 2013 SP2 (May 12): It’s hard for a 20-year-old product to keep on innovating as it did in the past. Exchange 2016 is on the way, but in effect it’s really a service pack (albeit a large one) for Exchange 2013. I really believe that there’s goodness in this approach, if only because of the continuing large-scale transfer of technology from the cloud to on-premises software.

Also published on WindowsITPro.com in May 2015 was an article describing the support of Office 365 Groups by Outlook 2016 (preview – build 4027). Groups are an interesting and valuable new entity only found in Office 365 whose use has been constrained by the lack of support by Outlook. Now it’s arrived, but there is still work to be done.

Why the power of Office Graph and Delve frightened some Ignite attendees (May 8): There’s no doubt that the Office Graph database is a huge unifying influence across Office 365. But the problem seen by some is in the amount of signals that the Graph gathers. There’s just too many. And some of that data might be misused, at least in the eyes of the privacy advocates. Reasonable fear or nothing to worry about?

Microsoft declares war on Symantec Enterprise Vault and looks to bring back data into Exchange Online (May 7): The new Office 365 Import service allows tenants to gather up PSTs and send the data (on SATA drives or over the network) to Microsoft, whereupon the data is ingested into Azure and made available to be imported into Office 365 mailboxes. All sounds good, except for Symantec and other third-party archive vendors, whose market seems to be contracted as Microsoft pursues a campaign to “bring back data into Exchange mailboxes”… Should be an interesting battle to observe.

News about Data Loss Prevention for SharePoint Online revealed at Microsoft Ignite (May 6): More coverage from the Ignite conference, this time describing how Microsoft is implementing the Data Loss Prevention feature into SharePoint Online and OneDrive for Business. All to stop users messing around with sensitive data.

News from Ignite: How Exchange 2016 benefits by technology transfer from the cloud (May 5): Microsoft took the opportunity at the Ignite conference to provide a lot more information about Exchange 2016. The biggest impression was not created by what seems to be a rather paltry list of new features but rather on quite how much technology is being transferred from Exchange Online to its on-premises counterpart.

Roadmap reveals potential for Office 365 Groups (May 5): As might be understood from the amount of coverage that I have afforded to this topic, Office 365 Groups are a big thing at the moment and they received a lot of air time at the Ignite conference. This piece covers the roadmap laid out by Microsoft at the event.

Satya Nadella launches Microsoft Ignite (May 4): In the longest blog post I have ever written for “Exchange Unwashed”, I covered the massive and everlasting (or so it seemed) keynote for the Ignite conference. I am still numb at the thought of quite how long it went on for, but at least some of the content and most of the announcements were pretty interesting and gave a solid pointer to the direction in which Microsoft is now heading.

Now on to the sultry month of June. No conferences to attend, but still lots of work to be done.

Follow Tony @12Knocksinna

Posted in Cloud, Email, Exchange, Exchange 2013, Office 365 | Tagged , , , , , , , , , | Leave a comment

Write some code and you can influence DAG failovers (for now anyway…)


A recent debate on the Exchange 2013 (unofficial) Facebook group started off with the question “can I built my own failover criteria in a DAG?” and pointed to the TechNet page on Active Manager.

The debate began with sheer denials, mostly on the basis that it didn’t seem to make sense for someone to attempt to second-guess the Exchange development engineers who have been working on this problem for many years. As the erudite Boris Lokhvitsky remarked: “In your car, do you have the desire to modify the combustion sequence or rearrange the valves in the engine so that it would run faster?”

In fact, Exchange 2013 evolved the failover criteria used by Exchange 2010 to take account of server health when Active Manager makes a decision about what target server to select to host a failing database in BCSS, or “best copy and server selection.”

But after a while, the esteemed Scott Schnoll weighed in to say that there is a way because Exchange accommodates a method called an Active Manager Extension, part of the third-party replication (TPR) API that exists in both Exchange 2010 and Exchange 2013. The TPR allows storage vendors to write their own continuous replication code and then stitch it together with the rest of the DAG components so that everything works together seamlessly. At least, that’s the theory.

TechNet says: “Exchange 2013 also includes a third-party replication API that enables organizations to use third-party synchronous replication solutions instead of the built-in continuous replication feature. Microsoft supports third-party solutions that use this API, provided that the solution provides the necessary functionality to replace all native continuous replication functionality that’s disabled as a result of using the API. Solutions are supported only when the API is used within a DAG to manage and activate mailbox database copies.”

On the surface, TPR seems like a wonderful idea. But the sad fact is that only EMC has ever implemented TPR in a solution called “Zero-Loss Protection for Exchange”, where they distinguish between “Native Database Availability Groups” (the normal kind) and “Synchronous Database Availability Groups” (the kind you’d use with an EMC CLARiiON SAN). The EMC Replication Enabler for Exchange is the component that leverages TPR.

I’m sure that EMC was very excited when Microsoft told them about the TPR because it must have seemed like a great way for EMC to defend their SAN installed base at a time when Microsoft was telling customers that they were engineering Exchange to exploit low-cost storage solutions. Since then the evidence is that not many people have actually used EMC’s solution and no other storage company appears to have been too interested in taking on the cost to develop and maintain their own replication solution for a DAG.

Indeed, given the hype around JBOD-type storage for Exchange, especially in the two years since Microsoft shipped Exchange 2013, anyone who proposed building a third-party replication solution for expensive SANs might be regarded as a candidate for lying down in a cool dark room until the idea passed. Even EMC is quite on the topic of using their code with Exchange 2013 and I imagine that the Replication Enabler is heading to the great byte wastebasket soon, if it hasn’t already reached there.

So Scott was right in his assertion that there is a way for someone to affect the way that Active Manager handles database failovers. You simply have to crack open your favorite IDE and write the code to leverage TPR. Simple. Just like that. Or maybe not. But the bad news is that your code will only work for Exchange 2010 and Exchange 2013 because Microsoft announced their intention to deprecate the API at the recent Ignite conference. It seems that Exchange 2016 will be the last version to support DIY DAG failovers.

As for me, I think I’ll let the Exchange developers take care of how replication happens inside DAGs. It just seems easier all round.

Follow Tony @12Knocksinna

Posted in Exchange, Exchange 2010, Exchange 2013 | Tagged , , , , , , | Leave a comment

Using PowerShell to convert Exchange Distribution Groups to Office 365 Groups


At last week’s Microsoft Ignite conference, I had the chance of attending a session called “Evolving distribution lists with Office 365 Groups” (a recording of the session is available online). The session described the integration with Outlook 2016 (no plans exist to back port the technology to Outlook 2013) and then went on to investigate how Microsoft views Office 365 Groups as a better alternative to old-style distribution groups. There’s no denying this fact. Distribution groups have been around since the dawn of email and haven’t evolved too much since. The last time Microsoft did anything to improve matters was the introduction of dynamic distribution lists in Exchange 2003.

Of course, the big issue with Office 365 Groups is that they will only ever live in the cloud. Microsoft is not going to incorporate them into the on-premises version of Exchange. You’ll be able to synchronize Office 365 Groups with on-premises Exchange via AADConnect if you operate a hybrid environment, but you won’t be able to create these groups on-premises.

One reason why this is so is the position that Office 365 Groups are moving to occupy within the Office 365 ecosystem as a single entity that permits access to many different forms of data available within the service. Today, membership of an Office 365 Group allows a user to access a shared mailbox, shared calendar, shared OneNote notebook, and a document library. The signs are that more resources will be accessible in the future, all granted through group membership.

Anyway, if you want more information about Office 365 Groups, you can read it in chapter 7 of “Office 365 for Exchange Professionals”.

Speaking of which, keeping the content of an eBook about Office 365 requires you to pay a lot of attention to what Microsoft is saying to customers at conferences such as Ignite. You never know when a speaker provides some information that should be included in the book or requires a change to the book’s content. In this case, I was interested in how Alfons Staerk approached the topic of migration from old-style distribution groups to Office 365 Groups.

It looks very much as if customers will be left to their own devices to migrate distribution groups as they wish. Of course, distribution groups bring their own complexities to the table. How should you deal with nested groups, for instance. What do you do with groups that include mail-enabled public folders, mail contacts, and shared mailboxes in their membership as none of these objects are supported in Office 365 Groups. And, of course, Office 365 Groups use their own mechanism to access resources across different parts of the service, so what do you do with mail-enabled universal security groups?

Microsoft demonstrated a program called “Hummingbird” that will soon be available (when the lawyers are happy) that can migrate a distribution group to an Office 365 group, subject to the caveats expressed above. Apparently the source code of the program will also be available to allow you to do your own thing.

But the approach taken to migrate a distribution group with a PowerShell script was more interesting. Up to recently, the PowerShell support for Office 365 Groups was just plain bad. You couldn’t create a new group or update group membership, both of which seem like fundamental operations. Microsoft is currently rolling out a new set of cmdlets to Office 365 tenants that address the problem. These are the *-UnifiedGroup cmdlet set to maintain group objects and the *–UnifiedGroupLinks cmdlet set to maintain group membership.

The script shown by Alfons was rudimentary but effective, but only for very simple distribution groups whose membership is solely composed of mailboxes. Everyone loves a challenge, and I decided that it would be a good thing to learn more about how to use the new cmdlets to work with Office 365 Groups, so I set about working on the ConvertDLtoO365Group.ps1 PowerShell script. After all, we need to bring out a second edition of “Office 365 for Exchange Professionals” (probably in September) that should cover this topic.

I’m no programmer. At least, I haven’t been for many years. My COBOL and VAX BASIC skills are rusty now but, as Jeffrey Snover keeps on reminding me, the joy of PowerShell is its ability to put things together bit by bit until something really good is done. PowerShell is like Lego bricks in that respect.

I hacked my way through several versions of the script. The current version is available in the TechNet gallery for anyone to download (and hopefully improve). The script runs in the context of a PowerShell session that is already connected to Exchange Online and does the following:

  • Takes the alias or name of a distribution group as the input parameter.
  • Performs some initial checks to see whether the distribution group exists or an Office 365 Group with the same alias exists. And that it’s an object of type MailUniversalDistributionGroup, which is the only type we can convert.
  • Checks the members of the input group to strip out those that can’t be added to the target Office 365 Group.
  • Checks whether the input group has member join restrictions. If it has (the group is “Closed” or “ApprovalRequired”), the admin is prompted to decide whether they want to create a private Office 365 Group. You can’t currently change the group type, so this is an important decision.
  • Tells the admin what’s happening and asks to proceed.
  • The new Office 365 Group is created with the New-UnifiedGroup It uses the same alias as the input distribution group because a new alias is given to that group.
  • As many of the properties of the input distribution group as possible are moved to the new Office 365 Group (not all can be because there isn’t a direct 1-to-1 mapping between the two object types). The Set-UnifiedGroup cmdlet is used for this purpose.
  • In particular, the Office 365 Group is set to auto-subscribe new members so that it mimics the distribution of new content via email as members expect from a distribution group.
  • The membership is added to the new Office 365 Group using the Add-UnifiedGroupLinks
  • Group members can be in three sets of links (owners, members, and subscribers). Because the new Office 365 Group is intended to behave like an email distribution group, the members are added to the members and subscribers sets.
  • The owners/managers of the input distribution group are added as owners of the Office 365 Group.
  • The email address of the input distribution group is switched to the new Office 365 Group so that new traffic goes there.
Converting an Exchange Distribution Group with ConvertDLtoO365Group.ps1

Converting an Exchange Distribution Group with ConvertDLtoO365Group.ps1

Phew! That’s a lot of processing. There are some known issues. For example, the –AccessType parameter for the New-UnifiedGroup cmdlet doesn’t work at present, so only public groups can be created. This is a known bug and is being fixed by Microsoft. Another issue is that running Add-UnifiedGroupLinks to add mailboxes as subscribers doesn’t work. This bug is also known and a fix will be available shortly.

However, the point is that it’s a PowerShell script and because it’s a script the code is there and available for all to see – and hopefully improve.

Thanks to Alfons Staerk and Sam Koppes of Microsoft for their encouragement. I think they quite liked seeing the demo script shown by Alfons take on a life of its own…

Enjoy!

Follow Tony @12Knocksinna

Posted in Cloud, Exchange, Office 365 | Tagged , , , , , , , | Leave a comment

Ignite 2015 wrap-up: Very much a V1.0 product


Chicago

I’m just back from the Microsoft Ignite conference in Chicago. With over 23,000 people attending, it was a huge event and the central concourse reminded me of a very busy railway station from time to time. Microsoft has announced that they are going to run Ignite in Chicago in 2016 (May 9-13) and I bet they’ve received a ton of advice from attendees that will help to make the 2016 event better.

The most common points I have heard expressed were:

Stop the shouting! Microsoft hired a company called EventForce to run the logistics and EventForce hired a bunch of college students to help attendees get from place to place. Because the McCormick Place conference center is so large this is a necessary service, but it would have been nice if some coaching had been provided so that the “purple shirts”  and the other security staff didn’t feel the need to shout at attendees. At times, it felt like cattle queuing to enter a milking shed. Attendees pay good money to attend Ignite; they don’t need to be insulted.

Fix the Wi-Fi. It’s a terrifically difficult challenge to provide a fast and robust Wi-Fi service for so many devices, but we’re talking about the world’s largest software company here and a company that is capable of running massively scalable infrastructures in the cloud. The service was awful in many places for Monday and Tuesday and seemed to improve as the week progressed. It would be nice if it worked from the start.

Make the buses run all day. The conference center is equipped with a hotel (the Hyatt Regency), at least two Starbucks, and a McDonalds. That’s not much choice if you want to duck out of sessions for a couple of hours to refresh yourself or do some work in private. But the bus service to the hotels stopped during the middle of the day, forcing attendees to take a cab or walk 3 or more miles back to the center of Chicago. On the upside, the people running the bus service were friendly and helpful and the buses use a special road to get back to downtown and so avoid the need to grapple with most of the traffic.

Provide edible food. No one expects great food at a conference. Imagine how difficult it is to prepare and serve meals for so many people in the 90 minute break assigned for lunch. Now make it more difficult by serving food that is attractive and edible. But people pay for the food in their conference fee and it was not good to find so many attendees voicing opinions that Ignite served the very worst food ever experienced at a major conference. That’s not the kind of thing you want to hear.

Shorten the keynote. Microsoft is in a quandary here. Ignite brings together many different groups, all of whom want to hear something about their own area of technology. So the keynote seemed to be an attempt to satisfy everyone in the audience and it lasted forever. I sat down at 8:30AM and escaped at 11:50AM. And that was too long. The keynote contained lots of great content and Julia White proved to be a real hit with her contribution, but it lasted way too long. And the annoying warm up act of the two talking heads who pratted on inanely before the keynote started did my head in.

PerryIgnite

Rationalize and streamline the scheduling. The conference center is big and room are spread apart. Many attendees seemed to be in perpetual motion to get from place to place and some clocked up many miles on their tracking device of choice (unsurprisingly, the Microsoft Band was noticeable by its popularity). Microsoft wanted to provide an event that substituted for TechEd, MEC, the SharePoint Conference, the Lync Conference, and others, so why didn’t particular areas of the conference center become fulcrum points for these communities by grouping related sessions together? It would have been nice not to have to see every floor and every area in the center. On the other hand, exercise is goodness.

Let speakers propose sessions they want to give. Asking experienced conference speakers to make themselves available for a session to be assigned to them is not a good way to create a compelling agenda. It’s an interesting idea that didn’t work. I think it’s just fine for Microsoft to assign sessions to its employees to ensure that the content it wants covered is delivered, but the same tactic doesn’t work with external speakers. Let us do our thing and cover technology with passion… and let’s do more “unplugged” sessions too. These work really well as long as they are moderated correctly.

More beer please. I didn’t attend the conference celebration because having dinner with 23,000 others and experiencing the lines for food, bathrooms, and beer didn’t seem like something I would enjoy. But I heard from many that the beer ran out early. Clearly not a good thing to happen in an event catering for IT types.

On a positive note, any of the sessions I attended were of the standard that I expected. This is only one person’s view and I heard from quite a few attendees that they were disappointed with the marketing-like content of some sessions. It’s also hard to come up with a schedule that everyone likes and I know some were disappointed when attractive sessions clashed or the schedule meant that fewer could get to a session than you’d expect. The Exchange compliance session was one of this – an interesting topic with huge expertise on the panel, attended by just a few people.

But you’ll get ups and downs at any conference and Microsoft provides an excellent Channel 9 service to allow attendees and people who weren’t at the event to catch up on all the sessions present at Ignite.

I also liked the Skype service provided to allow attendees to call home from the conference center. It was a small but nice gesture that leverages one of Microsoft’s products.

Ready to distribute copies of

Ready to distribute copies of “Office 365 for Exchange Professionals” with Technical Editor Jeff Guillet

And of course, we had the chance to launch our “Office 365 for Exchange Professionals” eBook at Ignite. Microsoft did a very nice job of producing 200 printed copies of the book that were given away to attendees. They also provided 400 copies on USB for my “Bumps and Blips” session on Thursday. And we were supported too by ENow Software, TransVault, Binary Tree, and NetMail, all of whom had special versions of the book for distribution to their customers. Copies of the book are available online. Because it’s an eBook that needs to be kept current with the evolution of the service, we will be updating it with a goal of releasing a new version at IT/DEV Connections in Las Vegas next September.

Best of all was the chance to meet up with so many people working in the industry. I’ve never seen so many Microsoft engineers at a single conference (expected because all of the other conferences are now gone); there were a huge number of great industry luminaries floating around (some at several feet off the group); lots of MVPs attended; and the exhibition hall – although very large – always had someone interesting to talk with or something interesting to investigate.

Overall, Ignite 2015 reminded me very much of a Microsoft V1.0 product. Obvious goodness and a great platform to build off, but lots of places to improve. I will be back in May 2016 for V2.0.

Follow Tony @12Knocksinna

Posted in Cloud, Technology | Tagged , | 10 Comments

New management cmdlets for Office 365 Groups


Several of the sessions about Office 365 Groups given at Microsoft Ignite in Chicago have addressed the obvious deficiency in management controls that exist in the initial implementation. Part of this was the total inability to create new groups or remove existing groups with PowerShell. You can run Get-GroupMailbox to return the properties of a group and Set-GroupMailbox to set some properties, but that’s hardly enough to satisfy the needs of those who want to manage groups.

The answer is the new set of *–UnifiedGroup and *-UnifiedGroupLinks cmdlet sets that are now rolling out to Office 365 tenants. These cmdlets provide a far more comprehensive set of controls over groups and replace the old cmdlets.

So we can now run New-UnifiedGroup to create a new Office 365 group and Remove-UnifiedGroup to clean up afterwards. Or even Set-UnifiedGroup to maintain group properties if we want to keep it. For example, here’s the command to create a new group object in Azure Active Directory.

New-UnifiedGroup –DisplayName “Ignite 2016” –Alias “Ignite2016” –EmailAddresses Ignite2016@contoso.com

The group is created as an empty object. You have to use the Add-UnifiedGroupLinks cmdlet to add members to the group. First, let’s add an owner:

Add-UnifiedGroup –Identity “Ignite2016” –LinkType Owners –Links TRedmond

To add a member:

Add-UnifiedGroup –Identity “Ignite2016” –LinkType Members  –Links JGuillet

A subscriber (who receives updates by email) can be added by changing the LinkType to “Subscribers”

This stuff isn’t fully documented yet because the roll-out is still in progress. Part of that roll-out is a transformation of existing groups from their existing Azure Active Directory object type to a form that can be synchronized back to on-premises Active Directory. This will allow hybrid Exchange deployments to access Office 365 Groups – the groups will remain in the cloud but they will show up in the on-premises GAL and will be treated as a distribution group. You’ll need to use AADConnect to synchronize AD with AAD to make this all happen.

You’ll see some evidence of the transformation in your tenant by running the Get-UnifiedGroup cmdlet. If this doesn’t return anything, you know that the existing Office 365 groups have not yet been transformed in AAD. When the groups are processed, they will show up when you run Get-UnifiedGroup, as will any new group that you create with New-UnifiedGroup.

You can also see the difference by running the Get-Recipient –RecipientTypeDetails “GroupMailbox” command. This will return all the old and new Office 365 Groups. The old groups have a recipient type of “UserMailbox” while the new groups use “MailUniversalDistributionGroup”, just like regular Exchange distribution groups. This is the change that allows cloud-to-on-premises synchronization of Office 365 groups.

For more information about Office 365 groups, see chapter 7 of “Office 365 for Exchange Professionals” where the topic is dealt with in some depth.

Follow Tony @12Knocksinna

Posted in Office 365 | Tagged , , , , , , | 12 Comments

Understanding how Exchange Online runs


Understanding how Office 365 operates is always an interesting challenge because Microsoft usually doesn’t say too much about how things work within the service. But the Exchange Online folks are pretty good at telling what they are up to, which brings us to the “Behind the Curtain: How we run Exchange Online” session at Microsoft Ignite in Chicago, featuring the talents of Vivek Sharma (Director of Office 365 product management) and Perry Clarke (VP of Exchange development). This provided an update to a similar session given at MEC in 2014 and is available on Microsoft’s Channel 9 service.

Both speakers are interesting people in their own right. In the past, Vivek had a lot to do with the implementation of PowerShell in Exchange 2007 and since then has focused on bringing Exchange Online through from the initial beginnings to BPOS to where it is today. Perry is one of Microsoft’s deep thinkers. A conversation with him is likely to explore what we’ll all be doing in five years’ time and it’s obvious that he has some pretty solid ideas on that point. He wrote the foreword for our just-published “Office 365 for Exchange Professionals” eBook and said some nice things about us, much to the amusement of some of the members of the Exchange development group.

The session began with some comments from Perry about how cloud services are changing the way people think about technology. Typically, companies look at three factors to assess a technology. Cost, risk, and user experience (or functionality). Perry maintains that the cost of an Exchange Online mailbox is at a point that no on-premises implementation can match, if costs are accurately calculated and everything is included. Part of this is because Microsoft has massive buying power for datacenters, storage, servers, and network to achieve price points that even the largest on-premises customer can only dream about.

A solid SLA track record (the most recent result was 99.99% for the first quarter of 2015) means that the perceived risk of companies putting their most important work on Office 365 is much less than it was four years ago when Microsoft launched the service. Finally, the functionality that can be delivered by a cloud service is so much ahead of what is possible for on-premises deployments because of the direct involvement of the engineering group (and some functionality, like Delve and Clutter is only available in the cloud). In a nutshell, Perry advanced a case that cloud services is the only way to achieve the desired combination of cost, risk, and functionality for a technology like email.

Returning back to the size and scale of Exchange Online, some data was offered to illustrate what Microsoft manages. It’s obvious from Microsoft’s financial results that they are enjoying growing revenue from commercial cloud services (the last quarter reported an annual growth of 106%). This growth is reflected in a 1350% increase in Exchange Online servers from Aug 1 2012 to 23 April 2015 compared to the 600% increase reported in 2014. The massive increase in servers is required to maintain capacity and to allow Microsoft to absorb new tenants who move to Office 365.

Interestingly, Exchange Online uses the same kind of rings to distribute new software. The rings are the developers, the Office 365 team, Microsoft in general, First Release Office 365 tenants, and finally, General availability. A similar approach is followed with the current Windows 10 insider program and is due to be used in the Windows Update for Business program announced at Monday’s Ignite keynote.

The increase in users means that Office 365 now deals with 55 billion client authentications annually. That kind of transactional volume cannot be handled when the infrastructure doesn’t scale efficiently.

Exchange Online uses 150 petabytes of storage, most of which is taken up by the 4 copies of the 1.2 million mailbox databases. The standard Office 365 mailbox quota is 50 GB, but naturally it takes time for users (maybe 90 million – Microsoft isn’t saying) to use this quota. The thought went through my mind of how many of the 8 TB 7200 rpm standard JBOD drives used by Office 365 fail daily and how they track and fix all the failures. The answer for how many is “a lot” and the management is done through a mixture of a very sophisticated service fabric and human intervention (to remove and replace the failed drives).

The service fabric controls and manages operations flowing across the service and deals with more than 500 million events that are collected hourly. In addition, 250 million synthetic test transactions are generated daily to validate that the Exchange Online service is working properly. The signals gathered by the transactions are analyzed by computers to detect and fix problems, just like the Managed Availability system in Exchange 2013. There’s no surprise here because Managed Availability is an obvious example of technology transfer from the cloud to on-premises (even more technology is being transferred in Exchange 2016). Machine learning is applied to correlate signals and compare them against known sets (that represent a satisfactory condition) to allow engineers to triangulate and identify the particular root problem.

An automation and orchestration workflow engine is used to maintain servers. The most common problems are hardware (disks and controllers), network, and software bugs. Problems can be automatically fixed or left to engineers, who can set off workflow items to address issues. Processes such as server deployments and upgrades are also dealt with through workflow in a way that allows Exchange Online to bring new capacity online within days of deciding that it’s needed. In this respect, new capacity means something like an additional 40 Database Availability Groups rather than a single server.

A DevOps model is used to run Exchange Online. In other words, development engineers don’t simply throw code over the wall to operations and then switch off. Instead, members of the development group up to and including VP level are on call to handle problems that arise in the service. This ensures that engineers take responsibility for the code that they write for the service. If they get it right, happiness and undisturbed nights. But if they get it wrong…

I find sessions that provide an insight into the trials and tribulations of operating a massive multi-tenant environment very interesting and worthwhile. Although you can follow them later online, there’s nothing quite like hearing someone speak in person. This session reinforced my view that something very special occurs to make Office 365 operate. Take the time to view it once the video is posted and make your own mind up.

Follow Tony @12Knocksinna

Posted in Office 365 | Tagged , , , , , | Leave a comment