Cloud-based email services such as Exchange Online bring a great deal of value to the table. A service delivered for a fixed known cost that is constantly refreshed with new features (“evergreen software”) and relieves administrators of the need to perform all the mundane tasks required for day-to-day server management. In addition, the provider takes responsibility for knitting together software to deliver new services, such as Clutter, Office Delve, and People View – all of which were previewed at the MEC 2014 conference and have recently appeared in “the service”.

And when all of this is backed up by an impressive workflow to automation operations (as explained in this “Behind the Scenes session” from MEC 2014), it seems like cloud-based email is the answer for almost everyone.

By their very nature, cloud operations have to proceed within strict limits as this is the only way that automation can be applied to make everything happen in a predictable and robust manner. If automation can’t be applied to a task, you need human intervention and that’s bad because it’s a) expensive and b) prone to error. Neither of these characteristics contribute to the successful economic operation of massive services.

The Office 365 design and support team do their level best to ensure that very few situations occur that cannot be handled by their procedures. And in the general run of things, they do very well, which is why Microsoft can report better Office 365 results every quarter. More customers are in the cloud, more companies are using Microsoft cloud services, and more servers, datacenters, and network have been deployed to satisfy demand. It’s all a happy picture.

That is, of course, until you enter the realm of problems that are not catered for by cloud operations, which is when you realize just how much control you cede when your mailbox is absorbed by the cloud.

Take backups for example. Now, I know that the mere mention of backups is sufficient to make some readers curl up in a ball because they’ve been scarred when bad things happened with backups in the past. Actually, the backups probably worked or seemed to work; problems usually happen when you attempt to restore data. But in most cases, companies have their backup and restore regimes worked out and they’re happy.

But backup happens – or rather doesn’t happen – in a different way when you deal with the scale found in Office 365. Just think about how you would do backups for the 100,000 servers that run Exchange Online. It’s a nightmare just to plan backups, let alone figure out how to handle the resulting backup media.

So Microsoft doesn’t do backups for Exchange Online. And why would they? After all, there’s plenty of technology built into Exchange to allow backups to be eliminated, or so the story of Exchange native data protection goes. As explained by TechNet, Exchange Online uses multiple database copies in DAGs arranged across multiple datacenters to ensure that nothing goes wrong and data is protected. We also learned at MEC that Exchange Online uses lagged database copies to ensure that creeping corruption can’t cause total meltdown. It all sounds wonderful. And automation implemented in features such as the Replay Lag Manager makes this technology work better for on-premises customers too.

It is, as long as you want to recover data that is still around. That means that the required items are still in a database because once items are permanently removed from a database, they are gone forever. Remember – there are no backups.

By default, single item recovery is enabled for mailboxes to ensure that the Managed Folder Assistant won’t delete items until the deleted items retention period lapses. The default period is 14 days, which isn’t a lot. You can increase the deleted items retention period to a maximum of 30 days whereas on-premises Exchange allows a maximum of 24,855 days. The on-premises maximum seems a little ridiculous as it allows for more than 68 years of deleted rubbish to accumulate in a mailbox. On the other hand, 30 days seems pretty restrictive.

With the maximum set, a user has 30 days to recover a deleted message. If that period passes and a user then “remembers” that some important item has been deleted, then they are plumb out of luck and no manner of pleading to the Office 365 support desk will cause them to budge. In any case, what could support do? It’s inconceivable that they would take the lagged database copy and use it to recover the item even if it was possible to do so (i.e. the lagged period was sufficient so that the item was not yet deleted in the lagged copy). Such an operation would be intensely manual, expensive, and potentially compromise the smooth operation of the service.

All you can do to protect the mailboxes of important people who might delete important mail in a forgetful moment is to put their mailboxes on litigation hold. This is not a good answer because it’s not what litigation hold is designed to do, but it works.

The alternative is to educate users not to do stupid things. After all, they have a massive mailbox quota so why would they want to delete anything… or so the story goes. But users are humans and humans make mistakes and education might not work.

Or, you can do what Microsoft is proposing to do in the feature described in the Office 365 Roadmap which says:

“The default 30-day retention period of deleted items folder on an Exchange Online mailbox will now be removed.  This means the user no longer has to worry about their deleted items folder automatically deleting emails every 30 days, but instead they can choose to empty the folder at their convenience. The admin can set a limit through Exchange Admin Console and PowerShell if they want to set a default limit on the folder.”

In fact, I think the text is incorrect. What seems to be happening is that Microsoft is removing the retention tag for the Deleted Items folder from the Default MRM Policy that is automatically applied to every Exchange Online mailbox, or they are disabling the tag for the Deleted Items folder (by setting its RetentionEnabled property to $False). This would be the smarter course of action as the tag can then stay in place on items but will be ignored by the Managed Folder Assistant.

In either case, the net effect will be that items will accumulate in the Deleted Items folder for up to two years, after which the default move to archive tag will kick in and the items will be moved to an archive mailbox, if one is enabled. If not, the items will simply remain in the Deleted Items folder until the user decides to empty that folder. The items will never get into the Recoverable Items folder and so the deleted items retention period described above won’t affect their recoverability. The extra storage really doesn’t matter because most users will accumulate no more than a gigabyte or two of deleted items annually.

An item on the Office 365 roadmap means that Microsoft is working on its delivery for sometime in the future. So far the exact time for the implementation of this feature is uncertain, but it should come in the relatively near future.

If you consider that it is better for the Deleted Items folder to be cleared out on a regular basis, you can re-enable the Deleted Items retention tag. Alternatively, you can adopt a middle course and update the Deleted Items retention tag to lengthen its retention period. Something like a 120 retention period seems reasonable. If someone has not worked out that they have lost something important after four months, that information might not be so important after all.

Conceptually, in some respects, the on-premises scenario for item recovery is much simpler (for the user). Administrators locate backup media for a time when the item was known to be in the database, the backup is restored to a recovery database and the item is then transferred to a PST. The PST is then cheerfully handed over to the user with the best wishes of the administrators. All is well. Apart, that is, from the number of people hours consumed in this activity. And that’s why Exchange Online uses Native Data Protection and trades large amounts of disk space to ensure that the need to recover items for users simply doesn’t exist.

Native Data Protection is part of what you sign up to when you move mailboxes to the cloud. Like other elements of the cloud experience, you have to trust the operators to keep your data safe. Outside the unique requirements of forgetful users, the cloud works. Not maybe the way that you’d like it to work, but it does work.

Follow Tony @12Knocksinna