The amiable Kanika Ramji, program manager for Exchange public folders, might have gulped a little when she saw the packed crowd at the “Experts Unplugged” session covering Exchange’s longest-lasting and most-persistent collaboration technology at the Microsoft Exchange Conference (MEC) on April 1. After all, the Internet had buzzed with criticism after Microsoft revealed the scalability limitations that afflict modern public folders and much of the passion surrounding that issue had transferred into the room.

After the session I think Kanika was more optimistic because the discussion never descended into bickering and hostility. Sure, some strong words were said, but the net outcome of the session (which proved the wisdom of including this kind of interaction at MEC) was extremely positive. The development group went away with clear marching orders as to what should be done: fix the scalability problems, OWA support for calendar and contact public folders, and better management and reporting tools.

The discussion about scalability was interesting. Microsoft couldn’t but admit that the 10,000 public folder limit was ridiculous. Many people in the room represented companies with over 100,000 old-style public folders and hard questions were asked about how Exchange 2013 could have shipped without the scalability limits being known.

It’s hard to change a software architecture to something radically different. New-style public folders hold their content in public folder mailboxes instead of dedicated databases. There’s nothing wrong here because Exchange databases are very scalable and high performing. The issue is all to do with the public folder hierarchy and specifically, how updates have to be made to the primary copy of the hierarchy, which is automatically created in the first public folder mailbox in an organization (a good reason to get this aspect of your deployment correct).

In a busy public folder deployment, it’s quite likely that folder updates happen regularly. Each update has to be referred back to the primary public folder mailbox and then rippled out to all of the other mailboxes, each of which holds a copy of the hierarchy. The current model works well inside small deployments but runs out of steam as the number of folders (and therefore the likelihood of more updates) scales up. Microsoft set the current limit at 10,000 folders because they know that everything works at this threshold. Pass it and you run into increasing instability as the primary copy of the public folder hierarchy struggles to cope with inbound changes and the subsequent updates back to the hierarchy copies.

Microsoft says that they want to get the limit up to 1,000,000 folders. A reasonable amount of engineering effort and (possibly more important) test and validation will be needed to get them to that point. Until they do, customers who want to move large numbers of public folders to on-premises Exchange 2013 servers or Exchange Online in Office 365 will simply have to wait. No timescale was promised but reassurance was given that this is a top-priority work item. We shall just have to wait and see.

One thing is for sure. Microsoft understands that the credibility of their attempts to modernize public folders and to reassure customers, all of whom have used Exchange for a very long time, is at stake. They have to fix the scalability problems this time round. No other option exists.

And while Microsoft is working at improving modern public folders, perhaps they’ll also fix the other horrible flaw that exists in the implementation – the fact that if you lose the mailbox containing the primary hierarchy, no method is available to transfer responsibility for hierarchy updates to one of the secondary copies currently exists. That doesn’t sound like a highly available solution and it’s a huge and gaping hole in the current implementation. It’s funny how the passing of time and the pressure of real customer deployments exposes all the flaws in computer systems.

Follow Tony @12Knocksinna