XOORG, Edge and Exchange 2010 Hybrid

Posted on 2 CommentsPosted in 2010, Edge, EOP, exchange, exchange online, Exchange Online Protection, Exchange Server, Office 365

So you have found yourself in the position of moving to Exchange Online from a legacy version of Exchange Server, namely Exchange 2010. You are planning to move everyone, or mostly everyone to Exchange Online and directory synchronization plays a major part (can it play a minor part?) in your plans. So you have made the option to go hybrid mode when you discover that there are manual steps to making Exchange 2010 mail flow to Exchange Online work if you have Exchange Edge Servers in use.

So, what do you do. You look online and find a number of references to setting up XOORG, but nothing about what that is and nothing about what you really need to do. And this you found this article!

So, how do you configure Exchange Server 2010 with Edge Servers, so that you can have hybrid mode to Exchange Online.

Why You Need These Steps

So you ran the hybrid wizard, and it completed (eventually if you have a large number of users) and you start your testing only to find that emails never arrive in Office 365 whilst your MX record is still pointing on-premises. After a while you start to get NDR’s for your test emails saying “#554 5.4.6 Hop count exceeded – possible mail loop” and when you look at the diagnostic information for administrators at the bottom of the NDR you see that your email goes between the hub transport servers and the edge servers and back to the hub transport servers etc. and about three or so hours after sending it, with the various timeouts involved, the email NDR arrives and the message is not sent.

The problem is that the Edge Server sees the recipient as internal, and not in the cloud, as the email has been forwarded to the user@tenant.mail.onmicrosoft.com, and Exchange 2010 is authoritative for this namespace. You are missing a configuration that tells the Edge that some emails with certain properties are not internal, but really external and others (those coming back from the cloud) are the only ones to send internal to the on-premises servers.

So what do you do?

Preparation

Before you run the hybrid wizard you need to do the following. If you have already run the wizard that is fine, you will do these steps and run it again.

  1. Install a digital certificate on all your Edge Servers that is issued by a trusted third party (i.e. GoDaddy, Digicert and others). The private key for this certificate needs to be on each server as well, but you do not need to allow the key to be exported again.
  2. Enable the certificate for SMTP, but ensure you do not set it as the default certificate. You do this by using Exchange Management Shell to Get-ExchangeCertificate to key the key’s thumbprint value and then running Enable-ExchangeCertificate –Thumbrint <thumbprintvalue> –Services SMTP. At this point you are prompted if you want to set this certificate as the default certificate. The answer is always No!
  3. If you answer yes, then run the Enable-ExchangeCertificate cmdlet again, but this time for the certificate thumbprint that was the default and set the default back again. If you change the default you will break EdgeSync and internal mail flow for everyone. And you must use the self-signed certificate for EdgeSync and this third party issued certificate for cloud mail flow, as you cannot use the same certificate for both internal and external traffic.
  4. The certificate needs to be the same across all your Edge Servers.
  5. If you are doing multi-forest hybrid, then the certificate is only the same across all the Edge Servers in one Exchange Organization. The next organization in your multi-forest hybrid needs to use a different certificate for all its Edge Servers.
  6. Then take this same certificate and install it on a single Hub Transport server on-premises. The hybrid wizard cannot see what certificates you have on the Edge Servers, so you need to help the wizard along a bit. Again, this certificate needs enabling for SMTP, but not setting as the default certificate.

Running The Hybrid Wizard

Now you can run the hybrid wizard. The important answers you need to include here are that the hub transport server that you pick must be the one that you placed the certificate on, as you cannot pick the Edge Servers that you will use for mail flow in the wizard. But you will need to enter the IP addresses that your Edge Servers are published on the internet as, and you will need to enter the FQDN of the Edge Servers as well.

Complete the wizard and then time for some manual changes.

Manual Changes

The hybrid wizard will have made a send connector on-premises called “Outbound to Office 365”. You need to change this connector to use the Edge Servers as the source servers. Note that if you run the hybrid wizard again, you might need to reset this value back to the Edge Servers. So once all these required changes are made, remember that running the wizard again could constitute an unexpected change and so should be run with care or out of hours.

Use Set-SendConnector “Outbound to Office 365” -SourceTransportServers <EDGE1>,<EDGE2> and this will cause the send connector settings to replicate to the Edge Server.

Next get a copy of the FQDN value from the receive connector that the hybrid wizard created on the hub transport server. This receive connector will be called “Inbound from Office 365” and will be tied to the public IP ranged of Exchange Online Protection. As your Edge Servers receive the inbound emails from EOP, this receive connector will serve no purposes apart from the fact that its settings are the template for your receive connector on the Edge Servers that the wizard cannot modify. The same receive connector will also have a setting called TlsDomainCapabilities and the value of this setting will be mail.protection.outlook.com:AcceptOorgProtocol. AcceptOorgProtocol is the XOORG value that you see referenced on the internet, but it is really called AcceptOorgProtocol and this is the value that allows the Edge Server to distinguish between inbound and outbound mail for your Office 365 tenant.

So on each Edge Server run the following cmdlet in Exchange Management Shell to modify the default receive connector: Set-ReceiveConnector *def* -TlsDomainCapabilities mail.protection.outlook.com:AcceptOorgProtocol -Fqdn <fqdnFromTheInboundReceiveConnectorOnTheHubTransportServer>.

This needs repeating on each Edge Server. The FQDN value ensures that the correct certificate is selected and the TlsDomainCapabilities setting ensures you do not loop email to Office 365 back on-premises again. Other emails using the Default Receive Connector are not affected by this change, apart from now being able to offer the public certificate as well to their inbound partners.

You can now continue with your testing knowing that mail flow is working, so now onto AutoDiscover, clients, free/busy, public folders etc. etc. etc.

Exchange Edge Server and Common Attachment Blocking In Exchange Online Protection

Posted on Leave a commentPosted in 2007, 2010, 2013, 2016, Edge, EOP, exchange, exchange online, Exchange Online Protection, FOPE, IAmMEC, Office 365

Both Exchange Server Edge role and Exchange Online Protection have an attachment filtering policy. The default in Edge Server is quite long, and the default in EOP is quite short. There is also a few values that are common to both.

So, how do you merge the lists so that your Edge Server attachment filtering policy is copied to Exchange Online in advance of changing your MX record to EOP?

You run

Set-MalwareFilterPolicy Default -FileTypes ade,adp,cpl,app,bas,asx,bat,chm,cmd,com,crt,csh,exe,fxp,hlp,hta,inf,ins,isp,js,jse,ksh,lnk,mda,mdb,mde,mdt,mdw,mdz,msc,msi,msp,mst,ops,pcd,pif,prf,prg,ps1,ps11,ps11xml,ps1xml,ps2,ps2xml,psc1,psc2,reg,scf,scr,sct,shb,shs,url,vb,vbe,vbs,wsc,wsf,wsh,xnk,ace,ani,docm,jar

This takes both the Edge Server default list and the EOP default list, minus the duplicate values and adds them to EOP. If you have a different custom list then use the following PowerShell to get your two lists and then use the above (with “Default” being the name of the policy) PowerShell to update the list in the cloud

Edge Server: Get-AttachmentFilterEntry

EOP: $variable = Get-MalwareFilterPolicy Default
$variable.FileTypes

Secret NSA Listening Ports in Exchange Server 2013? Of Course Not…

Posted on Leave a commentPosted in 2013, Edge, exchange, firewall, IAmMEC, iis, networking, powershell, transport

But what do those extra ports in Exchange Server 2013 that are listening actually do.

If you bring up a command prompt on an Exchange Server 2013 machine and run netstat –ano | find “:25”. You will get back a list of IP addresses that are listening on any port starting 25. The last number on the line is the process ID for that listening port. So for Mailbox only role servers you are interested in the row that shows 0.0.0.0:25 and for a multirole (CAS and Mailbox) server, the row that shows 0.0.0.0:2525 as shown:

image

Above you can see that process 21476 is listening on 2525 and as this is a multirole server this process ID will be EdgeTransport.exe – you can verify this in Task Manager if you want.

Repeat the netstat cmd, this time for the process ID you have selected: netstat –ano | find “xxxxx” where xxxxx is the process ID for EdgeTransport.exe, as shown:

image

You will now see that EdgeTransport.exe is listening in on a range of ports for both IPv4 and the same ports for IPv6. These ports are 25 or 2525, which is used by 2007 or 2010 Edge role servers, or by other 2007/2010 Hub Transport role servers or by other 2013 Mailbox role servers or by 2013 CAS role servers to send emails to this server via SMTP. Port 465 is the port that 2013 CAS servers proxy authenticated SMTP connections to that they receive on port 587 from mail clients. But what about port 29952 in my example (and on your servers a different port) which changes each time the service is restarted?

If you do netstat again just for this port you will see something like the following:

image

This shows that nothing is connected currently to these ports, and so they seem to be doing nothing. But if on a different Exchange 2013 server you do some viewing of the transport queues then these ports will start to show some activity.

On a different server in my environment I ran Get-Queue –Server remoteservername. If I do this on the local server, then nothing special happens as Exchange does not need to connect to these ports, but if it is run from a different server and I ask it to show the queue on the first server that we have been looking at above, then these ports become used:

image

image

Above we can see Exchange Management Shell on mail5 connecting to mail4 (the original server in this blog post). The second picture from mail4 shows that port 29952 have received a connection from the IPv6  address of mail5 and specifically from port 65172 on that remote server.

If I look finally at the second server in this exercise and see what process is connecting from port 65172 (and again, your ports will be different) I see that process 15156 is doing this (the process ID is the last column in the output)

image

Taking this process ID to Task Manager, I see process 15156 is the IIS Worker Process, which is the process that PowerShell connects to to do its work.

image

Therefore, the random and changing port that EdgeTransport.exe listens on is nothing to do with PRISM, but all to do with your management of remote queues.

Moving Exchange Online Protection Junk Mail to the Junk Email Folder

Posted on Leave a commentPosted in 2007, 2010, 2013, active directory, cloud, Edge, EOP, exchange, exchange online, FOPE, mcm, mcsm, spam, transport

If you use Exchange Online Protection (EOP) to filter your email in the cloud and to remove spam and malware before onward delivery to you, and if you use Exchange 2007 or later on-premises, then you need to configure Exchange to move detected spam to the Junk Email folder in Outlook.

By default EOP detects two levels of spam (malware is automatically removed) and tags them. In Exchange, you need to use Transport Rules to move these emails to the users Junk Email folder. If you wish for EOP throw away junk emails of either detection level (i.e. high junk is discarded) then you still need to configure these transport rules to move the remaining detected junk email into the Junk Email folder in Outlook. If you place all EOP detected spam in the quarantine or throw it away, you should still create these transport rules as it means they are in place for any future changes you make at EOP.

If you use Exchange Online (part of Office 365) then you do not need to create these rules as they already exist (though you cannot see them in any admin tools you have).

The Transport Rules To Create

The following two transport rules need to be created on your Exchange organization. These two rules get created with the highest priority, moving all existing rules below them. They set the Spam Confidence Level of the message to 6 in these examples, though this should be set to a value that exceeds the SCLJunkThreshold organization wide setting for your Exchange Organization, as any email that exceeds this value is placed into the Junk Email folder upon delivery to the users mailbox.

New-TransportRule “Move EOP Detected Spam (SFV:SPM) to Junk Email Folder” -HeaderContainsMessageHeader “X-Forefront-Antispam-Report” -HeaderContainsWords “SFV:SPM” -SetSCL 6 -Priority 0

New-TransportRule “Move EOP Detected Spam (SFV:SKS) to Junk Email Folder” -HeaderContainsMessageHeader “X-Forefront-Antispam-Report” -HeaderContainsWords “SFV:SKS” -SetSCL 6 -Priority 1

Once these two rules are run, AD replication will need to be allowed to happen and then all Exchange Servers will action this rule for any email that EOP thinks is spam. The default value for SCLJunkThreshold is 4, so as long as the rules set the SCL value to greater than this value it should work. Use Get-OrganizationConfig | FL SCLJunkThreshold to get this value for your organization.

What the EOP Message Tags Mean

The X-Forefront-Antispam-Report header in the email contains many values. See http://technet.microsoft.com/en-us/library/dn205071(v=exchg.150).aspx for what EOP thinks about the spam rating of the message.

  • SFV – means Spam Filtering Verdict
  • SFV:SFE – Originated from a Safe Sender (EOP learns Outlook safe senders due to Windows Azure Directory Sync)
  • SFV:BLK – Originated from a Blocked Sender
  • SFV:SPM – Spam
  • SFV:SKS – (SKIP) The message was marked as spam prior to being processed by the spam filter. This includes messages where the message matched a Transport rule to automatically mark it as spam and bypass all additional filtering
  • SFV:NSPM – Not Spam

Removing Edge Subscription When Exchange 2013 Installed

Posted on 6 CommentsPosted in 2007, 2010, 2013, Edge, exchange, federation, IAmMEC, mcm, mcsm, smarthost, transport

Exchange 2013 does not have an Edge role (at the time of writing – Aug 2013). It is possible to use Exchange 2010 SP3 and install the Edge role should you need one.

There is a problem though when it comes to removing the Edge Subscription between an organization that contains Exchange 2013 servers and the Exchange 2010 Edge Server. To remove the subscription on the Edge server role you run Remove-EdgeSubscription servername and this removes both the subscription and any subscribed objects from the local AD LDS database on that Edge Server. But if any of these subscribed objects where created on Exchange 2013 after it was installed, then they will have an ExchangeVersion equal to 0.20 (15.0.0.0). The Exchange 2010 SP3 Remove-EdgeSubscription cannot process this object and so fails with:

Remove-EdgeSubscription : Can’t make this change because the object’s ExchangeVersion property is 0.20 (15.0.0.0), which is not supported by the current version 0.1 (8.0.535.0). You will need a newer version of Exchange to make this change. Property Name: ExchangeVersion
At line:1 char:24
+ Remove-EdgeSubscription <<<<  edge2 -Verbose
+ CategoryInfo          : NotSpecified: (0:Int32) [Remove-EdgeSubscription], DataValidationException
+ FullyQualifiedErrorId : D8A49A14,Microsoft.Exchange.Management.SystemConfigurationTasks.RemoveEdgeSubscription

The way to fix this is to find and manually remove the object with an ExchangeVersion of 0.20 (15.0.0.0) from the AD LDS database and then repeat the Remove-EdgeSubscription cmdlet – as that should now work (unless you have two or more objects with the higher version number to locate and delete).

  1. To find objects with an ExchangeVersion greater than “0.1 (8.0.535.0)”, which is the version Exchange 2010 will process, open ADSIEdit on the Edge server.
  2. Right-click the ADSI Edit node at the top of the window and choose Connect to…
    image
  3. In the Connection Settings dialog (shown above), change the Select a well known Naming Context to Configuration and type the local server name and the AD LDS port in the Select or type a domain or server field. The server:port value should be EDGESERVERNAME:50389
    image
  4. Expand the tree view until you reach CN=First Organization,CN=Microsoft Exchange,CN=Services,CN=Configuration,CN={AD258B4D-CCB4-4125-80C1-7B73CE066341}
    image
  5. You now need to look at each object, starting with those you remember having created since 2013 was installed, for an object who’s msExchVersion value is not 4535486012416. For example, in the below screenshot I have an accepted domain made due to Federation in Exchange 2013. This object (under CN=Accepted Domains,CN=Transport Settings,…) has a value of 88218628259840.
    image
  6. To validate that this is the correct object to manually delete, from the Exchange Management Shell on the Edge server, enter Get-Object | FT name,ExchangeVersion where Object is the cmdlet that you are looking to query – in my case it would be Get-AcceptedDomain.
    image
  7. As you can see, this object has a newer ExchangeVersion and so it is (at least) this object that is stopping Exchange EdgeSync from being removed.
  8. Manually delete this object in ADSI Edit (it is safe to do this as it will resync from the Exchange organization if you recreate Edge Subscription later). Do not delete it from the Active Directory with ADSI Edit – just from AD LDS. Take care to only delete this object and not the parent object.
  9. Once this object is gone, try Remove-EdgeSubscription servername again. If this is the only object, then the Edge Subscription will be removed in the Edge Server

You can now carry on with whatever it was that you were doing that required a removal of the Edge Subscription.