Highly Available Office 365 to On-Premises Mail Routing

This article looks at how to configure mail flow from Office 365 (via Exchange Online Protection – EOP) to your On Premises organization to ensure that it is highly available and work in disaster recovery scenarios with no impact. It is based on exactly the same principle to that which I blogged about in 2012: http://c7solutions.com/2012/05/highly-available-geo-redundancy-with-html on creating redundant outbound connections from Exchange on premises.

The best way to explain this feature is to describe it in the way of an example:

For example MCMEmail Ltd have Hybrid set up, and delivery to the cloud first. So the DNS zone for mcmemail.co.uk has MX pointing to EOP.

They then create a new DNS zone at either a subzone (as in this example) or a different domain if they have one available. In the example this could be hybrid.mcmemail.co.uk. Into this zone they add the following records:

10 MX oxford-a.hybrid.mcmemail.co.uk

10 MX oxford-b.hybrid.mcmemail.co.uk

20 MX nuneaton.hybrid.mcmemail.co.uk

The below picture shows an example of this configured in AWS Route 53 DNS (though there are other DNS providers available)

In Exchange Online Protection administration pages (Office 365 Portal > Exchange Admin > Mail Flow > Connectors and modify your on-premises connector to point to the new zone. Example shown in the below picture:

Then all email is always delivered to the Oxford datacentre and nothing to the Nuneaton one (where the DR servers reside) unless the two Oxford datacentres (A and B) are both offline and so the 10 preference does not answer at all. At that time and that time only does the 20 preference get connected to.

Posted

7 March 2014

2010, 2013, cloud, DNS, EOP, exchange, exchange online, Exchange Online Protection, hybrid, IAmMEC, MX, Office 365, smarthost, smtp

Brian Reid

Tags:

Comments

24 responses to “Highly Available Office 365 to On-Premises Mail Routing”

Ed Crowley

8 March 2014

Useful post. Thanks!

Reply
The UC Architects » Episode 35: Service Packed!

14 March 2014

[…] http://www.c7solutions.com/2014/03/highly-available-office-365-to-on-premises-mail-routing […]

Reply
filip

31 March 2016

Does this also require that the ssl cert used for SMTP must contain all mx fqdn? So I need 3 external ip, 3 mx record & san cert with 4 fqdn?

Reply
1. Brian Reid
  
  31 March 2016
  
  No. The certificate name is does not need to match the FQDN for SMTP. For the connectors you can specify the certificate CN value that must be presented by the server when Office 365 connects (Outbound Connector) or when you connect to Office 365 (Inbound Connector). If the certificate name is correct then the connection will work. This is the TlsCertificateName property of the connector. Note that TechNet says “The TlsCertificateName parameter specifies the X.509 certificate to use with TLS sessions and secure mail. Valid input for this parameter is [I]Issuer[S]Subject. The Issuer value is found in the certificate’s Issuer field, and the Subject value is found in the certificate’s Subject field”, but you can just use the subject name as well.
  
  If you do not specify TlsCertificateName then SMTP will use Opportunistic TLS, which means any certificate (valid or not) will work.
  
  Reply
  1. filip
    
    2 April 2016
    
    Thanks Brian, all clear about the cert.
    Unfortunately external ips are scarce so not always applyable, but love your solution.
    On another note can we use a NLB or would a LB adjust the TLS SMTP as a smarthost would do ? And if we use LB we need to add its ip to the receive connector and we miss the source ip and see LB ip instead. How do you fix this?
    
    Reply
  2. filip
    
    3 April 2016
    
    Brian, I found this TechNet article that suggests you put a transport fqdn on the certificate. Is this the fqdn you enter during the HCW ?
    
    https://technet.microsoft.com/en-us/library/hh563848(v=exchg.150).aspx
    
    Reply
    1. Brian Reid
      
      3 April 2016
      
      The value of the FQDN is the endpoint you connect to. The certificate subject alternate name for transport does not need to match this
      
      Reply
      1. filip
        
        4 April 2016
        
        Hi Brian,
        Sorry to bother one more…
        I found this TechNet article also, they mention EOP drops if the FQDN is not on the cert/san.
        
        https://technet.microsoft.com/en-us/library/jj659055(v=exchg.150).aspx
        
        When using forced TLS transport, the sending and receiving servers examine the certificate configured on the other server. The subject name, or one of the subject alternative names (SANs), configured on the certificates must match the FQDN that an administrator has explicitly specified on the other server. For example, if EOP is configured to accept and secure messages sent from the mail.contoso.com FQDN, the sending on-premises Client Access or Edge Transport server must have an SSL certificate with mail.contoso.com in either the subject name or SAN. If this requirement isn’t met, the connection is refused by EOP.
        
        NoteNote:
        
        The FQDN used doesn’t need to match the email domain name of the recipients. The only requirement is that the FQDN in the certificate subject name or SAN must match the FQDN that the receiving or sending servers are configured to accept.
      2. Brian Reid
        
        4 April 2016
        
        You are not doing “Forced TLS Transport” for hybrid. The wizard sets the inbound connector with the string that it expects to find in the certificate and is nothing to do with the FQDN of the domain. Then when on-premises connects to EOP, EOP validates the certificate matches the string and if so lets you in
      3. filip
        
        4 April 2016
        
        Could not find that in any Microsoft documentation (about the fqdn not needed on cert), is it documented anywhere or a glitch in the Microsoft documentation 🙂
        Is it advisable to put the HCW FQDN on the cert or do you consult/implement a certificate with only autodiscover and webmail.companyname.com on the certificate?
        
        So hybrid mail flow is Opportunistic TLS and not Forced TLS?
        
        Does this mean o365/connector checks the certificate string and if it’s on the on-prem connection it’s accepted and uses secure smtp? and does this secure smtp use port 25 or port 587 ?
        Do we need to publish/nat 25 and 587 on the firewall to the on-prem exchange servers?
      4. Brian Reid
        
        4 April 2016
        
        Hybrid mail flow is TCP 25 only. And you only need to publish the Exchange Servers you list in the hybrid wizard.
        
        Hybrid mail flow sets the TLSCertificateName property on the connector and will only deliver if the Send Connector on-premises TLSCertificateName value matches the certificate offered by the receive connector in EOP. The reverse is also true, the Outbound Connector in EOP needs to have the TLSCertificateName value set to the subject name of your on-premises certificate (as selected in the hybrid wizard).
        
        This is not Opportunistic TLS (which will downgrade to plain text if no certificate is offered [no STARTTLS]) or Forced TLS where the certificate must match the FQDN of the server. The FQDN of the server is irrelevant for hybrid mail flow.
      5. filip
        
        4 April 2016
        
        Cristal clear Brian!
        So you can even use a trusted 3rd party cert with a complete other fqdn then exchange uses cause the CN is used, interesting.
        
        Thanks for all your time, explanation & patience.
        Will you be attending/present on Ignite?
        If yes I’ll buy you a beer/pint 🙂
      6. Brian Reid
        
        4 April 2016
        
        Yes, I’ll be at Ignite. Not sure if presenting yet!
      7. Dave
        
        7 August 2017
        
        Hi Brian, you mentioned below – The reverse is also true, the Outbound Connector in EOP needs to have the TLSCertificateName value set to the subject name of your on-premises certificate (as selected in the hybrid wizard).
        
        I have set TlsCertificateName and have found that the receive connector (on Edge server) is still accepting mail from clients with no certificate. Any advice? was looking to restrict access to the valid certificate (along with IP restriction)
        
        Thanks
        Dave
      8. Brian Reid
        
        10 October 2017
        
        If you use just the IP restriction, which the hybrid wizard does in later versions of Exchange, then you dont need the certificate restriction as well
Brian Reid

2 April 2016

Yes, you can use a load balancer. But unless you do “direct server return” feature of the load balancer you will see the IP of the load balancer. The DSR feature will be called different things in different makes of load balancer.
A load balancer would not adjust the certificate (layer 4 load balancing), but would if layer 7 load balancing. Typically for SMTP you do layer 4 (TCP).

Reply
1. filip
  
  2 April 2016
  
  Brian, interesting and also thanks for pointing that layer 7 LB breaks “X-MS-Exchange-Organization-AuthAs”.
  For sinlge external ip it’s best to go for layer 4 DSR
  It’s never easy in the Exchange world 🙂
  
  Reply
filip

2 April 2016

On another note, is it supported to use ssl offloading for all Exchange services in a hybrid environment?

Reply
Brian Reid

2 April 2016

SSL offload – no. MRS does not support offloading.
Regarding X-MS-Exchange-Organization-AuthAs I did not say Layer 7 load balancers changed anything. The load balancer should pass the data through untouched after inspection.

Reply
filip

2 April 2016

In Your previous post i read

A load balancer would not adjust the certificate (layer 4 load balancing), but would if layer 7 load balancing.

I understood ‘layer 4 not but layer 7 would’
So neither layer 4 &7 will touch x header?
Can we enable SSL offloading for ews but somehow leave SSL enabled for mrsproxy specific?

Reply
1. Brian Reid
  
  2 April 2016
  
  X-Header and certificate are not the same thing. Layer 7 and the certificate is in the load balancer, layer 4 and it is not. Headers nit modified at all.
  
  SSL offloading – not unless your LB allows it with complex rules.
  
  Reply
filip

2 April 2016

Kuddo’s to you Brian. Again my big thanks. I need some load balancer schooling. Life as Exchange admin is though 🙂
Enjoy your weekend!

Reply
Nick

21 August 2023

Brian,

Do you have any official documentation on this?

I had my send connector in EXO configured using an MX record composed of 2 A records in separate datacenters of equal weight.

When one DC was down message delivery to on-premise mailboxes was intermittent.

Message trace in EXO shows failure to the one MX record and no reattempts to the alternate MX record.

Microsoft support said I should configure multiple smart hosts in Exchange Online send connector instead.

Reply
1. Brian Reid
  
  4 October 2023
  
  So this article was written in 2014, so there is no guarantee it still works (I’ve certainly not tested it). If you create a single outbound connector with multiple smarthosts that will work.
  
  Exchange Online routes email and if the endpoint is unavailable it will mark the endpoint as such and recheck it intermittently (initially quickly incase of a network glitch, and then more infrequently like 15 or 30 minutes). Once the endpoint is back online, Exchange Server will reuse it. So if you have multiple smarthosts Exchange Online will only deliver to the ones that it thinks are online.
  
  The disadvantage of this is that it would route to both the primary and DR datacenters at the same time (as both are online). The MX record method (which though I have not tested I am sure is still working) relies on having two different preference values and not two of the equal weight (as in your example). There is no point doing the MX approach if you have equal weights as that means equal delivery, and this blog post is all about unequal delivery.
  
  Reply

Highly Available Office 365 to On-Premises Mail Routing

Comments

24 responses to “Highly Available Office 365 to On-Premises Mail Routing”

Leave a Reply to filip Cancel reply