My recent experience of working through a problem with Office 365 support confirmed once again just how fundamental the paradigm shift is for IT professionals tasked with resolving problems after workload moves to the cloud.
Overall, the support experience was good. The problem was solved. The people I dealt with were professional and polite and seemed to be more competent than earlier encounters. Microsoft provided the necessary systems to track progress of the problem and I was informed of how things were going at every stage. Microsoft even assigned me an “Advocacy Manager” to keep track of the case. So I felt pampered.
However, the same issues with cloud support persist. Progress towards resolution was too slow. Too many questions were reiterated for no apparent reason. Escalation was slow and the amount of value added by first-level support was not as high as I had hoped. But we got there in the end.
Supporting issues with cloud systems is difficult. Can you imagine working as a front-line support professional who has to deal with customer problems? Gathering the necessary information to form a mental picture of the problem is the first challenge. Understanding its impact on customer operations is another. Figuring out how to proceed to a timely resolution is a third.
Of course, Microsoft has databases that record details of support incidents, their symptoms, and eventual resolutions. Sometimes a quick search will reveal an answer and an incident can be closed quickly. In other cases, like mine, the symptoms are not so clear or a match cannot be found by searching, so the investigation process requires substantially more effort.
Events unfold at a very measured and controlled pace as Office 365 support personnel are obviously guided by some very clear instructions about how to gather information, interpret the data, and move towards resolution. Like many other aspects of Office 365, a structured playbook dictates how support unfolds and escalation from first-level to second-level staff happens. This is how it should be as you can’t afford to have different support personnel take radically different approaches to problem resolution in an environment that supports millions of end users. Such a scenario is a recipe for poor customer satisfaction and erratic problem resolution. It would also likely lead to higher support costs and dissatisfied customers.
It’s also important to understand that none of the frontline people working in Office 365 support have no access to datacenters. These people are responsible for collecting and refining information so that problems are well understood, resolved if possible, and then passed on to engineering if a solution cannot be found. Complex problems will probably involve a number of interactions between support personnel and engineering. In these cases, the support personnel retain responsibility for communication with the customer and engineering works in the background.
The highly structured approach to problem resolution means that cases often take longer to resolve than you might anticipate. The slowness in steps unfolding is often the most frustrating aspect for IT professionals. In the on-premises world everything is available and all components can be interrogated and tested in a drive towards a fix. With the cloud, you’re dealing with an amorphous blob where any control that you have terminates as packets leave your network en route across the uncontrollable Internet to a cloud datacenter. You can change client or device settings, but that’s it. Everything associated with server processing is in the cloud and hidden from you.
All of this means that IT professionals have a very different role to play during cloud support incidents. Instead of taking responsibility for pursuing a problem from client to server to resolution, your role is to gather information and provide it to cloud support. The better and more accurate the information you provide, the sooner the problem will be resolved. It is all too easy to lead cloud support down a dead end and even easier to spend days waiting for them to figure out that they are in a dead end. Responding to support with accurate and timely information is the only real way you have of controlling how quickly resolution will happen.
User communication needs increased attention from IT professionals too. You might understand that control over problem resolution has been ceded to cloud support but an end user is unlikely to understand the complexities and nuances involved in moving work to the cloud. All they know is that their email doesn’t work or they can’t get to a document in a SharePoint library. It seems pretty clear therefore that IT professionals have to do more work to reassure end users that their issues really are being worked.
Some might decry the transition of responsibility to the cloud. But similar changes have occurred before in both IT and elsewhere. We don’t have to load programs from paper tape now and cars are less maintainable than before because they use so many computers. You don’t see people worrying that they can’t use paper tape programs and few really mind the passing of fallible carburettors that tended to fail on damp mornings. We adapt and get on with the new situation.
Follow Tony @12Knocksinna
Thoughtsofanidlemind's Blog 
A very interesting post Sir. Thank you.
I have supported Exchange, in a number of large enterprises, for roughly 20 years; the first fifteen years were On-Premise and the last five years have been Hosted (where we’re the Hostee, not the Hoster). From my perspective (i.e. the ‘customer’ in the relationship) the support arrangement in a Hosted environment works much as you describe for Cloud services – so I have experienced, first-hand, the “transition of responsibility” that you refer to.
I’m sure that many on-premise Exchange admins are apprehensive about ceding control and responsibility to a third party. But over the years Exchange has become increasing complex and with the advent of DAGs, and high availability across multiple sites, all but the largest enterprises lack both the expertise and resources to ‘do Exchange right’. The physical resources alone – such as access to Tier 4 datacenters in at least 2 cities – will be prohibitive for many companies. Even if you co-lo in top tier datacenters, you still have the additional expense of duplicating your infrastructure (servers, storage, network) for full redundancy. Yes, there are plenty of highly skilled on-premise Exchange admins out there but, for the most part, even the best of them are not going to have the real-world experience that the large Hosters and Cloud providers have gained from architecting, implementing, and supporting millions of mailboxes. When large enterprises upgrade from say Exchange 2003 to Exchange 2010, they typically retain the same on-premise support folks, but there is a very steep learning curve and no amount of reading, training, and testing is going to compare to the real-world practical experience and knowledge already gained by the large Hosters and Cloud providers. The on-premise support teams are invariably doing their best – usually with limited resources – but they’re often forced into making compromises, such being compelled to use a shared SAN that is not properly managed and maintained. Those little compromises become the weakest links, and pretty soon you find that your ‘best practice’ on-premise implementation is more like a house of cards.
As you point out, in Cloud environments (and I’d add Hosted to that), the admins job is feed accurate information to the service provider, and to keep the internal customers informed. If you were previously an on-premise admin, you may be very familiar with how troubleshoot problems, but you need to be wary of complicating the situation. Yes, by all means do some investigation, but know when to step back and hand it over to the folks who are ultimately responsible for the service. Besides, why keep a dog and bark yourself? 😉
The bottom line is that the large Hosters and Cloud providers can do it better, and often cheaper.
(Though, as you’ve pointed out, if you have more than 100,000 Public Folders, Cloud is not yet an option. And there are clearly other challenges to overcome too)
Your car analogy is a good one. Most people these days rely on the dealer – or at very least a qualified mechanic – to service and maintain their cars, especially if they are relying on that car to make a living. Sure, if you’re an enthusiast, you may enjoy tinkering with your vintage muscle car at the weekends (and it sure is a blast to drive), but on Monday morning you’ll take your Camry to work.
@Drew
You are working at a Cloud/Hosting vendor so OF COURSE you are selling Cloud/Hosting. Your argument against On-Premises Admins are NOT valid & sounds BS 🙂 Such as “there is a very steep learning curve and no amount of reading, training, and testing is going”.
The issue with Cloud/Hosting is the Vendor OWNS the companies’ data after the move to the Cloud/Hosting. Also we hear every day about Backdoor Access (NSA). Companies WILL keep data in-house & CAN train On-Premises Admins………….
@Joe
I apologize if original post wasn’t clear:
“… the last five years have been Hosted (where we’re the Hostee, not the Hoster). From my perspective (i.e. the ‘customer’ in the relationship) …”
So, to clarify: I work for a Fortune 500 company in the Energy sector, and our 25,000+ mailbox Exchange environment is hosted by a 3rd party.
Also, having spent 15 years supporting large on-premise Exchange deployments with over 50,000 mailboxes, I have first-hand experience of both on-premise and Hosted/Cloud environments.
The learning curve between some versions of Exchange is very steep and even the most diligent administrators cannot expect to pick up all the technical subtleties (some of which may not even be undocumented) via training, reading, and testing. No amount of classes, or books (event Mr. Redmond’s masterpieces) are going to compare to real-world experience – which the Hosters, and Could providers have in abundance because they’re already managing vast numbers of mailboxes. Does that make them perfect? No. But, unarguably, they have a wealth of real-world experience.
And no, the Hosters and Cloud providers do not “own” your data – you do.
Security is, of course, a concern for all IT professionals. There are really 3 aspects to security that relate to this discussion:
1) Hackers, and people trying to steal intellectual property.
I think a strong argument can be made that the large Hosters and Cloud providers do a much better job of protecting their environments and data than 99.9% of the regular companies with on-premise Exchange.
2) Data surrendered to the authorities in response to formal legal requests.
These are unavoidable, whether the data is in the cloud or on premise. Yes, in the past, maybe the Hosting/Cloud companies were a little too willing to simply hand the data over to the authorities, but, in the post-Snowden word, they are pushing back.
3) Government snooping.
Regrettably, this is going to happen either way – especially when you’re talking about email.
Of course many companies will prefer to keep Exchange on-premise, and that is their prerogative. The pros and cons of the cloud can be debated elsewhere, and perhaps I strayed a bit too far into that debate in my comments, but Mr. Redmond’s original was post was regarding the support experience *when* you are in the cloud – not whether you *should* move to the cloud.
@Drew
Unfortunately you are 100% wrong when you say “the Hosters and Cloud providers do not own your data”. The Data is residing on the Hosting/Cloud providers’ servers, so the Hosting/Cloud provider OWN the data and they CAN do whatever they want with the data 🙂
Also you are forgetting Backdoor Access (NSA PRISM) to the Public Cloud.
I agree with Joe.
The Hosting/Cloud vendor OWNS your data once your data reside on Hosting/Cloud provider’s servers. The Hosting/Cloud provider has full access and control over your data.
@Lee
Yes, if you are using the word “owns” in the sense that the Hosting/Cloud providers ‘have full access to’ you data, then yes, you are correct. Much like your bank has full access to your money. Does your bank “own” your money?
But, just as your banks business model depends on keeping your money safe, the Hosting/Cloud providers business model depends on keeping your data safe. Microsoft, for example, is pretty much betting the company on the Cloud; they simply *must* get it right. I agree that the cloud providers are not there yet, but, in the post-Snowden era, they know the stakes could not be higher and they are scrambling to address the concerns.
@Joe
Clearly I am not forgetting NSA PRISM; I specifically mentioned “Government snooping” in my comments.
@Drew
If you have control and access over something you pretty much OWN it. If you have control and access over my PC you pretty much OWN my PC cuz you can control it and have access to it now who cares I bought the PC, same goes with Data. In Hosting/Cloud access and control is given to the vendor so the vendor pretty much OWN your data.
Pingback: Cloud Services Mean | MPLS