It was nice to read the memories of ex-Microsoft Distinguished Engineer Hal Berenson in a post about SQL 2014’s delayed durability feature. Like me, Hal used to work at Digital Equipment Corporation (DEC), where he specialized on database engineering. He tells the story of prototyping delayed durability to fix some performance issues in Rdb/VMS in 1986. As explained in TechNet, delayed durability is accomplished by “transaction log records are kept in a buffer and hardened to disk when the buffer fills or a buffer flushing event takes place.” As it turns out, the prototyped code was removed from Rdb/VMS and never used in production.
The description of how delayed durability works should be pretty familiar to anyone who knows anything about the Exchange Information Store and the way that ESE, the database engine, handles transactions. Exchange captures information about transactions in the current transaction log and only commits them to the database when the complete transaction is available.
As Hal explains in his post, after he left DEC and joined Microsoft, he helped the Exchange 4.0 development team address some performance problems that they had in the Store. MAPI was the only client protocol in use at the time and MAPI just loves properties when it comes to processing email. Each adjustment to a property (and there could be 40 or more made to process a message) was an individual transaction with a subsequent impact on performance as each transaction was logged. Hal suggested that the ESE team use delayed durability and the problem was resolved. Exchange 4.0 shipped with the feature in March 1996.
MEC attendees can get the latest information about how the Information Store and ESE combine to deliver robustness and durability by going to “The Latest on High Availability and Site Resilience” at 10:45am on Tuesday. And if you’re at MEC, why not come along to my 8:30 session on Wednesday when I talk about the wonders of Retention Policies. What could be a better start to the day?
I just love war stories like this. An idea that didn’t work out for a DEC product eventually helped to solve a major performance issue for Microsoft’s first enterprise email server and is now in SQL Server 2014. Time flies when you’re having fun…
Follow Tony @12Knocksinna