One or two NICs?


During the development of Exchange 2010, Microsoft originally required that any mailbox server participating in a Database Availability Group (DAG) had to be equipped with at least two NICs (Network Interface Controllers). One NIC handles “MAPI” or client traffic; the other handles the replication traffic generated by Exchange to keep database copies updated within the DAG.

Not everyone appreciated Microsoft’s stance because it meant that additional cost was imposed on smaller installations where you might want to run highly available mailbox servers but didn’t want to buy new hardware or update servers with additional NICs. I know that there seems to be a disconnect here because you might think that anyone interested in high availability would be willing to invest in the hardware required to do the job, but there are situations where companies who have run configurations such as Single Copy Clusters for Exchange 2003 or Exchange 2007 (the traditional “WolfPack” approach) for small offices now want to upgrade to Exchange 2010 and use a DAG and want to reuse their servers.

In any case, Microsoft retreated from their original position and you can operate mailbox servers within a DAG with just one NIC. In this configuration, all client and replication traffic are handled by the one NIC and everything works very nicely.

The question that is commonly asked therefore is what point you need to move to multiple NICs for an Exchange 2010 mailbox server? I think there are a number of points that deserve consideration:

  • The number of users supported by the server: Normally, a company feels more of an impact if a server that supports thousands of users fails than one that supports just a few hundred. If you operate small sites (for example, distributed offices), you probably configure relatively small servers and may decide that you will standardize on a single mailbox server design that is used everywhere. The prospect of a NIC failure is probably less common than a disk or other failure, so you decide to run just one NIC. On the other hand, if you’re operating large servers to support thousands of users within a centralized datacenter, you’re probably going to take every step that you can to prevent a hardware failure affecting service to users and it makes sense to equip the mailbox servers with multiple NICs and the additional cost will be accepted.
  • The true desire for high availability: People talk a lot about their desire to assure high availability and then throttle back as the factors necessary to protect the infrastructure such as cost and complexity climb. Mailbox servers within an Exchange 2010 DAG operate very well with just one DAG and you might decide that this server configuration will deliver a satisfactory amount of high availability for your infrastructure.  On the other hand, if email is deemed to be a critical application that is fundamental to the continuing success of the business, you’ll probably invest in the extra hardware.
  • The potential for outages: This factor is unique to each organization. You know how good your operations staff is, what kind of monitoring is in place to detect server and other outages, how old and well-maintained the equipment is, and other environmental factors such as network reliability and so on. If you operate brand-new hardware that has been well bedded-in before deployment and your operations and management processes are capable of dealing with outages swiftly and effectively, you’ll probably be happy to run with a single NIC (of course, your brand-new hardware might well be delivered with multiple NICs so the point then becomes moot). If not… well, the comfort factor of multiple NICs might be nice.

You’ll notice that the word “probably” is used a lot in this discussion. That’s because this issue is intensely unique to a company.  Companies have different cultures and part of that is a varying attitude to risk and cost that will lead them to make their own hardware design decisions. The only hard fact is that Exchange 2010 mailbox servers function well within a DAG whether they have a single NIC 0r multiple NICs. It’s only when problems occur and a server has a NIC failure that the other factors come into play and you may appreciate the additional flexibility afforded by multiple NICs.

I suspect that the server vendors are figuring out what kind of solutions they should sell to satisfy customer demand for highly available Exchange 2010 servers and that they will take the NIC issue into account as they determine suitable configurations. I think that we will see some OOTB HA Exchange 2010 hardware solutions appear in 2011 and it will be interesting to see how the server vendors elect for their configurations. In the meantime, if you have the choice, it’s best to be flexible and opt for multiple NICs as it removes a potential area of concern for your mailbox servers.

– Tony

Advertisements

About Tony Redmond ("Thoughts of an Idle Mind")

Exchange MVP, author, and rugby referee
This entry was posted in Exchange 2010 and tagged , , , . Bookmark the permalink.

7 Responses to One or two NICs?

  1. Constantino Tobio says:

    In our environment, it was a matter of necessity to set up our Mailbox servers to run on a single NIC, as our DAG spans two physical locations on different subnets. We could have set up a private replication network, I suppose, but it didn’t seem worth the trouble to set up and route a private network if the single-NIC method is perfectly viable.

  2. Same with my design…enviroment was running VMWare so NIC redundancy was already taken care off. Adding extra routes and VLAN to my second datacenter was not an option.
    To bad that almost every document states the replication network. Thanks for this post it comfirmed my own toughts!

  3. Paul van Eijk says:

    We are running our Exchannge 2010 Environment on VMWare vSphere 5 and we came across this question as well. Single NIC or Multiple NICS? Because we can only add a virtual NIC on the Exchange 2010 I think for us it doesn’t give more security for NIC failures. What are your oppinion on this, Tony?

    • I think that I would have more servers than less if they are virtualized and have to use virtual NICs. That way you spread the effect of any possible failure as much as you can. DAGs are always better when they have more servers than less…

      TR

      • Paul van Eijk says:

        Thanks Tony for your quick response. We have two virtual CAS servers (CAS Array) with HLB, two virtual Hub Transporters and two virtual Mailbox Servers for 2000 Mailboxes. Those two Mailbox Servers will have DAG and we are planning a dataware house for an Exchange Mailbox Server with enough diskspace for a second DAG. So I think that part have been covert

        Because in Vmware vCenter we have teamed 4 physical NICS to assign Virtual NICS to VM’s I think it doesn’t make sense in terms of NIC failures to have Multiple Virtual NICS in Exchange 2010. Are you sharing the same thoughs?

        Paul

      • I think that the first thing is to increase the number of mailbox servers in the DAG. Two doesn’t really give you much redundancy. Three would be better. I wouldn’t put another DAG into operation. Keep all the mailbox servers in one DAG until you are met by a problem (like maximum number of servers exceeded or poor network links) that forces you to create a second DAG.

        I agree that it doesn’t seem to make sense to have multiple virtual NICs that are tied back to a limited set of physical NICs. But maybe there’s a real VMware expert out there who can provide the definitive answer.

        TR

  4. Paul van Eijk says:

    Thanks for the advise of keep all in one DAG and sharing your thoughs about this single / multiple NIC(s) Tony

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s