The nice people at Microsoft Learning, previously famous for their decision to cancel the well-respected (but probably in need of reform) Microsoft Certified Architect (MCA) program, have scored another triumph with the MCSE recertification program.

The recertification program allows MCSEs to watch a number of Microsoft Virtual Academy (MVA) modules online and take the module-level assessments to prove that they understand the material that was presented. The results are then returned to MSL, who verify the data before they recertify the candidate.

MCSE: Messaging recertification requirements

Like many ideas, the recertification program seems good in theory. However, the effectiveness and worth of the program in action requires good MVA content backed up with solid assessments. For those who want to recertify as MCSE: Messaging, that means watching and passing 15 modules. The material doesn’t cover Exchange 2016 specifically, but seeing that Exchange 2013 and Exchange 2016 are so close, that shouldn’t be an issue.

Many program managers are featured in the video content, which is a good thing as they are talking about the product that they have helped to build. Of course, it’s all theory because the discussion focuses on how the product should work rather than how it does work in practice, but that’s always the case when a vendor presents material. What struck me is that the MVA material is a tad aged at this point as a lot of it was produced in 2013 and 2014, meaning that it does not reflect the current state of the art. Exchange on-premises has moved on through cumulative updates (and the release of Exchange 2016). Microsoft reported that 450 changes had been made inside Office 365 in the year to August 2015, which gives some insight into how quickly material and guidance changes.

The two Brians from the Exchange product group wax lyrical about implementation of Exchange 2013

I guess that material of this nature is always going to lag behind. It would have to be refreshed every quarter to have a chance of staying current, and that’s not going to happen. In any case, you can sit down with your beverage of choice and start watching videos and taking the module-level assessments. It will take time. Lots of time.

You’d hope that the questions posed by MVA to test the competence of those who participate in the recertification exercise are up to a certain standard of accuracy. Unfortunately, it seems like they are not. For example:

What feature is available in Exchange Online that is not available in Exchange on-premises?

1. Mail Tips
2. Mailbox delegation
3. Security Groups
4. Distribution Groups

This question is odd, to say the least. Both cloud and on-premises versions support A, B, and D, so C has to be the focus in the mind of those who set the question. However, Security Groups are available on both platforms and both support mail-enabled security groups. The question is what feature is available to Exchange Online that is missing on-premises but the answer is that the same features are available to both.

Or how about this?

Whoops! A figure too much…

Unhappily, the feature name is document fingerprinting. Figure printing is quite something else. I think it involves people covering their bodies in paint before rolling around on a canvas. Or something like that.

Tests are meant to examine the depth of knowledge that exists in an individual. Unfortunately, some of the test questions put to people who want to be recertified are based on marketing and a fair amount of Microsoft Kool-Aid. Here’s an example. Pick four answers from four, all of which are marketing cliches.

The Kool-Aid is very obvious here

An example of how questions suffer from ageing is shown below. The answer to question 5 is marked as incorrect, but the answer is actually correct because Microsoft updated mailbox auditing for Exchange Online last year to support owner action logging. This is not the fault of the instructor as the response was accurate in June 2015 when the module was published. It’s just wrong now.

Whoops – only 6 right out of 7, but that wrong answer is actually right…

Overall, the quality of the questions doesn’t give you much faith in the precision of the recertification process. Apart from the absolute howlers described above, it’s obvious that some work needs to be done on an ongoing basis to ensure that the questions are not outdated and invalidated due to the rapid pace of change that occurs inside the products for which people are being certified.

Another oddity about this process is that you do not have to watch the videos if you don’t want to. You can skip through all the steps in a module to get to the assessment and answer the questions there. And you can try your luck answering the questions several times until you are able to pass the assessment. While you’re going through the assessment questions, no possibility exists that anyone would have another browser tab open so as to be able to search for the answers.

In summary, you can recertify your MCSE: Messaging accreditation by watching a lot of video and answering some questions that can be a little odd at times. The quality of the process shines through in all respects.

Creating and operating high-quality accreditation programs is hard, especially for high-volume global audiences. Providing knowledge through the MVA is a worthy initiative that provides valuable information to those who care to watch, but the content presented for MCSE: Messaging is now aged and is too Microsoft-centric. It would be nice to see some  more field practitioners used as instructors.

Given the issues with the MVA content and questions, I don’t think that this route is fit for the purpose of recertification and wonder about just what achieving recertification actually means for individuals. Some are folks who will simply be upgrading their skills off a broad base of practical knowledge and experience and this will simply be the cherry on top of the cake for them. But I worry that “paper MCSEs” might be able to trumpet that they have all the skills and experience in the world required to take on complex projects because they’ve managed to be recertified due to their expertise in video watching and skill in navigating odd questions.

Of course, this wouldn’t be the first time that I disagreed with MSL and I suspect it won’t be the last…

Follow Tony @12Knocksinna