Speaking at TechEd Europe 2014

Posted on 4 CommentsPosted in certificates, cloud, EOP, exchange, exchange online, Exchange Online Protection, GeoDNS, hybrid, IAmMEC, journaling, mcm, mcsm, MVP, Office 365, smarthost, smtp, starttls, TechEd, TLS, transport

I’m please to announce that Microsoft have asked me to speak on “Everything You Need To Know About SMTP Transport for Office 365” at TechEd Europe 2014 in Barcelona. Its going to be a busy few weeks as I go from there to the MVP Summit in Redmond, WA straight from that event.

image

My session is going to see how you can ensure your migration to Office 365 will be successful with regards to keeping mail flow working and not seeing any non-deliverable messages. We will cover real world scenarios for hybrid and staged migrations so that we can consider the impact of mail flow at all stages of the project. We will look at testing mail flow, SMTP to multiple endpoints, solving firewalling issues, and how email addressing and distribution group delivery is done in Office 365 so that we always know where a user is and what is going to happen when they are migrated.

Compliance and hygiene issues will be covered with regards to potentially journaling from multiple places and the impact of having anti-spam filtering in Office 365 that might not be your mail flow entry point.

We will consider the best practices for changing SMTP endpoints and when is a good time to change over from on-premise first to cloud first delivery, and if you need to maintain on-premises delivery how should you go about that process.

And finally we will cover troubleshooting the process should it go wrong or how to see what is actually happening during your test phase when you are trying out different options to see which works for your company and your requirements.

Full details of the session, once it goes live, are at http://teeu2014.eventpoint.com/topic/details/OFC-B350 (Microsoft ID login needed to see this). Room and time to be announced.

Highly Available Office 365 to On-Premises Mail Routing

Posted on 22 CommentsPosted in 2010, 2013, cloud, DNS, EOP, exchange, exchange online, Exchange Online Protection, hybrid, IAmMEC, MX, Office 365, smarthost, smtp

This article looks at how to configure mail flow from Office 365 (via Exchange Online Protection – EOP) to your On Premises organization to ensure that it is highly available and work in disaster recovery scenarios with no impact. It is based on exactly the same principle to that which I blogged about in 2012: http://c7solutions.com/2012/05/highly-available-geo-redundancy-with-html on creating redundant outbound connections from Exchange on premises.

The best way to explain this feature is to describe it in the way of an example:

For example MCMEmail Ltd have Hybrid set up, and delivery to the cloud first. So the DNS zone for mcmemail.co.uk has MX pointing to EOP.

They then create a new DNS zone at either a subzone (as in this example) or a different domain if they have one available. In the example this could be hybrid.mcmemail.co.uk. Into this zone they add the following records:

10 MX oxford-a.hybrid.mcmemail.co.uk

10 MX oxford-b.hybrid.mcmemail.co.uk

20 MX nuneaton.hybrid.mcmemail.co.uk

The below picture shows an example of this configured in AWS Route 53 DNS (though there are other DNS providers available)

image

In Exchange Online Protection administration pages (Office 365 Portal > Exchange Admin > Mail Flow > Connectors and modify your on-premises connector to point to the new zone. Example shown in the below picture:

image

Then all email is always delivered to the Oxford datacentre and nothing to the Nuneaton one (where the DR servers reside) unless the two Oxford datacentres (A and B) are both offline and so the 10 preference does not answer at all. At that time and that time only does the 20 preference get connected to.

Domain Secure and Edge Servers

Posted on Leave a commentPosted in 2007, 2010, 2013, certificates, cloud, exchange, firewall, smarthost, smtp, transport

I was asked a question recently on the Microsoft Certified Master course for Exchange 2010 and was told that the answer was not clearly written up on the internet. So I thought I would write this blog post. The question was based on the idea that Domain Secure worked from a Hub Transport server in the classroom lab but not when mail flow went via an Edge server.

Domain Secure is end to end security, it cannot have anything in the middle – i.e. it cannot go via an Exchange Edge server, an Exchange 2013 Frontend Server or a third party SMTP relay.

The SMTP client in the connection (the send connector host) needs to connect to the SMTP server (the receive connector host) and swap certificates and prove the other side is who the other side say they are – i.e. mutual authentication. Also the domains must match the TLS list in TransportConfig (TLSSendDomainSecureList and TLSReceiveDomainSecureList). Therefore anything in the middle will offer a different certificate and so Domain Secure fails.

If there is a middle party and you want to do mutual authentication (i.e. swap certs to prove who you are), with one party offering their cert and not the cert of the final recipient domain (i.e. mail.messaging.microsoft.com or postini.com etc.) then use TLSAuthLevel and the DomainValidation option on the send connector (an SP1 addition to Exchange 2010). No green ticky ticky though.

Edge can do Domain Secure though. But Edge needs to be the starting point, i.e. the host of the send connector. So configure Domain Secure on the Edge (i.e. set the certificates and correct firewall settings) and ensure that the send connector for Domain Secure has the Edge server as the source. Ensure Edge Domain Secure receive connector is the target for inbound as well if you want it to work both ways. And of course you need working EdgeSync so hubs can deliver to Edge so that Edge can deliver emails for you.

Exchange 2013 Transport Agents

Posted on 3 CommentsPosted in 2007, 2010, 2013, agent, exchange, sdk, smtp, visual studio

Earlier today I posted http://blog.c7solutions.com/2012/10/creating-simple-exchange-server.html on how to create a transport agent in Exchange, and though the steps cover some of the detail for Exchange Server 2013 they do not cover some of the detail, so I’ve added that to this post below:

  • Use .NET Framework 4.0 for Exchange 2013 though it is possible to write the agents using earlier versions of the .NET Framework. If you do use earlier versions then you need to update some of the config files on the server to state the version levels that you support and Enable Support for Legacy Transport Agents on Technet lists the steps for this.
  • The Front End Transport role (which is installed on the CAS Server role) can support SMTP agents bound to events up to, but not including OnEndOfData.
  • The Hub Transport role (which is installed on the Mailbox Server role) can support all SMTP agents except for those that bind to the OnConnect event.
  • The Hub Transport role also support routing agents and delivery agents. The CAS role supports neither of these agent types.
  • The Exchange 2013 Edge Transport Server role (expected with Exchange 2013 SP1) will support agents at the OnConnect event.
  • To install a transport agent on a multi-role server (CAS and Mailbox on the same machine) then you need to use can use -TransportService Hub to the Install-TransportAgent cmdlet shown in the earlier blog if you want to bind the agent to the Hub Transport events. If you want to use the Front End Transport events (OnMailFrom to OnEndOfHeaders) then use –TransportService FrontEnd instead (updated 2 Oct from Philippe comment below).
  • To install a transport agent on a CAS only server role you need to use local PowerShell and load the Exchange Management Shell snap-in with Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010
  • Finally, to view the agents in a given service, always state the service name or you will get back the Hub Transport service by default. For example Get-TransportAgent –TransportService FrontEnd for the agents bound to the Front End Transport service. Other values are Hub, Edge and MailboxSubmission and MailboxDelivery (though the last two don’t seem to work in the pre-release version on Exchange Server 2013.

Creating a Simple Exchange Server Transport Agent

Posted on 5 CommentsPosted in 2007, 2010, 2013, agent, exchange, mcm, sdk, smtp, transport agent, visual studio

This blog post follows a session that I delivered at the MEC 2012 conference in Orlando. If you attended the conference the slides are available on http://mymec.mecisback.com for the rest of 2012.

Part of the transport agents session was writing a new transport agent, and the example agent was to do add a form of catch-all functionality. The example code below (which purposefully needs editing to work, as its designed to be a learning tool) takes an email in the form of alias_tag@domain.com where the final recipient is alias@domain.com and the subject line is changed to read [tag] Original Subject. Therefore email addresses can be given out that are an adjusted form of your real address, and on receipt of the email the correct recipient address is determined in code (the value before the _ appended to the domain name) so that you do not need to add lots of aliases to your user account in Active Directory. The subject is then changed to indicate the email came from this alternative original address. For example, an email to bill_linkedin@microsoft.com would go to bill@microsoft.com and have [linkedin] added to the start of the subject.

As this partial catch all transport agent needs access to the recipient information and the subject, the agent needs to be bound to OnEndOfHeaders or OnEndOfData in the SMTP stack. It could also be bound to OnSubmittedMessage as a routing agent (rather than an SMTP agent) but could not be bound to OnResolvedMessage as recipient resolution has already happened by this point.

Writing the Transport Agent

To create the transport agent you need Visual Studio – the basic version of Visual Studio are sufficient and you need to copy the two DLL’s from Program Files\Microsoft\Exchange Server\V14\Public (or the V15 folder for Exchange 2013) to a folder on your development machine. You need a copy of these DLL’s for every version/service pack/update rollup of Exchange that you will run your agent on, as the DLL’s might change between these versions, and if you have the wrong version you will not be able to install your agent on a new server or if the server is updated, the transport service will fail as it will not be able to load the agent.

In Visual Studio, create a .NET 2 or 3.5 Class Library (for Visual Basic if using the below code) for Exchange 2010 or .NET 4.0 Class Library for Exchange 2013. For the project name enter something descriptive for what you are going to create (rather then ClassLibrary1). For example MECDemoPartialCatchAll.

image

Copy and paste the following code into the class.vb file, replacing the template text.

   1: Imports System

   2: Imports System.Collections.Generic

   3: Imports System.Text

   4: Imports Microsoft.Exchange.Data.Transport

   5: Imports Microsoft.Exchange.Data.Transport.Smtp

   6:

   7: Namespace XXXXX REM Change This

   8:

   9:     NotInheritable Class YYYYYY REM Change This

  10:         Inherits SmtpReceiveAgentFactory

  11:

  12:         Public Overrides Function CreateAgent(ByVal server As SmtpServer) As SmtpReceiveAgent

  13:             Return New ZZZZZ REM Change This

  14:         End Function

  15:

  16:     End Class

  17:

  18:     Public Class ZZZZZ REM Change This

  19:         Inherits SmtpReceiveAgent

  20:

  21:         Private Sub MyEndOfDataHandler(ByVal source As ReceiveMessageEventSource, ByVal e As EndOfDataEventArgs) Handles Me REM Change This

  22:             ' Get and change the recipient from alias_tag to alias (only doing for 1 recipient for simplicity)

  23:             If e.MailItem.Recipients.Count = 1 And InStr(e.MailItem.Recipients.Item(0).Address, "_") > 1 Then

  24:                 Dim Recipient() As String = Split(e.MailItem.Recipients.Item(0).Address, "@", 2)

  25:                 Dim EmailAlias() As String = Split(Recipient(0), "_", 2)

  26:                 'EmailAlias(0) = alias to send email to

  27:                 'EmailAlias(1) = tag for subject line

  28:                 'Recipient(1) = domain

  29:

  30:                 ' The following line prepends [tag] to the subject of the message.

  31: REM Change This                 "[" + EmailAlias(1).ToString + "] " + e.MailItem.Message.Subject

  32:

  33:                 'the following drops the current recipient

  34:                 e.MailItem.Recipients.Remove(e.MailItem.Recipients.Item(0).Address)

  35:

  36:                 'the following adds the recipient back again, this time using the alias without the tag

  37:                 e.MailItem.Recipients.Add(New RoutingAddress(EmailAlias(0).ToString + "@" + Recipient(1).ToString))

  38:             End If

  39:         End Sub

  40:

  41:     End Class

  42:

  43: End Namespace

Download this code

The remaining steps on creating the agent will be to modify the lines above that are marked with REM statements and then to add the reference DLL’s you copied from your Exchange Server and finally build your DLL.

Visual Studio can tell you when your code contains errors before you build your code, but to do so you need to reference the DLL’s that you obtained from your Exchange Server earlier in this blog. To do this you need to have copied the two DLL’s to a unique folder on your computer. I use c:\temp\Agent Authoring\Dll\E14-SP2-RU4 to store the DLL’s from Exchange 2010, SP2, RU4. Then when RU5 is released, if the DLL’s have changed I will place them in an E14-SP2-RU5 folder and update the references in my project. If I keep using the same folder then Visual Studio does not refresh the DLL but uses an existing cached copy.

To add the DLL’s right-click the project name to the right of the Visual Studio window and choose Properties (or press Alt+Enter):

image

Change to the Reference’s tab and click Add > Browse to find and add these two DLL’s.

image

You should see the two DLL’s listed as well as the default references for DLL’s included in the class library template that you used in Visual Studio.

Back on the code, its time to make the changes. Each of the changes and why it is needed are detailed based on the line numbers above

Line 7 – Namespace

This value of the namespace for the transport agent. Standard conventions indicate that this should be your company name. Therefore in your code change XXXXX to C7Solutions

Line 9 – NotInheritable Class

This value, YYYYYY, is the class that you are creating. This class inherits all the functionality of the Microsoft.Exchange.Data.Transport.Smtp.SmtpReceiveAgentFactory class. For this partial catch-all agent, a good name would be PartialCatchAllFactory.

Line 13 and 18 – Public Class

The Public Class contains the code to execute when the agent is called. This class has a name (ZZZZZ in the sample code above) and should be given a name that represents what the code does. This name in this example can be PartialCatchAll and it needs to be used in Line 13 (inside the PartialCatchAllFactory code to indicate the code to call instead of SmtpReceiveAgentFactory. And of course it is needed on line 18 to name the actual block of code. The name to use for this example will be PartialCatchAll

Line 21 – Handler

This line is missing the end of the code. Its a line that says run the code in this subroutine if the OnEndOfData event is called. Change the end of the line to read Me.OnEndOfData. This value should be one of the suggestions in the drop down list if you have done everything correct so far:

image

This subroutine also returns e as a pointer to the email message (i.e. you can modify e.MailItem.DeliveryPriority and many other properties) and source as a reference to the SMTP connection (i.e. with source.Disconnect you would close the SMTP session).

In this code lines 34 and 37 change the recipient. Line 34 removes the first recipient and then line 37 adds a new recipient that matches the alias + the domain.

Line 31 – Modify Subject

Finally, this code is missing the start of line 31. You need to enter e.MailItem.Message.Subject =  at the start of the line so that they subject becomes [tag] + original subject.

Building the Transport Agent DLL

The Errors List at the bottom of the Visual Studio screen should be empty by now. So time to build the project. If you have not saved this project in Visual Studio, the DLL will be created in the %temp% directory, so best recommendation is to save the project before building the DLL.

To save the project click File >  Save All and enter a suitable name for the project.

image

The project name becomes the solution name by default and then click Save.

The final thing to check before you build the DLL is the Root namespace value. This should be set to the name of the class library and can be set during creation of the library. If you have written your library in VB.Net then you will need this value to register the DLL. If you used C# then you will not need this value. As we have used VB.Net above, we need to check the root namespace value. This can be found on the project properties page.

image

If your DLL is ready to release (i.e. you have build and tested it already) then choose Build > Configuration Manager menu. Change the Active solution configuration to Release and click Close. If you do not make this change then the DLL will be created in the project_name\bin\debug folder and its possible that Visual Studio does not compile all the optimizations to the code that it can. For production items that perform at as good a speed as you can write optimal code, you should change this to Release for the Configuration value. The DLL will be created in project_folder\bin\release.

To build your DLL (debug or release) select the Build > Build RootNamespaceValue menu. This will create a DLL in project_name\bin\debug or project_name\bin\release.  A working version for Exchange 2010 SP2, RU4 can be downloaded from here.

Installing the Transport Agent

Copy the DLL to all your Exchange Servers that have the hub transport role installed on them (or if this should only apply to emails inbound from the internet, then just the servers that are listed on your MX records or the first servers connected to for inbound emails). Place the file in a directory (say C:\C7Agents) and run the following five lines from Exchange Management Shell:

Install-TransportAgent -Name “C7PartialCatchAll” -AssemblyPath “C:\C7Agents\MECDemoPartialCatchAll.dll”
-TransportAgentFactory “MECDemoPartialCatchAll.C7Solutions.PartialCatchAllFactory”

Enable-TransportAgent “C7CatchAll”

Restart-Service MSExchangeTransport

IISRESET

# Recommend closing EMS window now

Note that this restarts the transport service (required) and IIS (which will effect OWA and other CAS roles on the same machine). Remote Powershell (an IIS resource) will lock the DLL open, and so if you need to delete the DLL after uninstalling it, you need to have reset IIS and closed the Powershell window.

Finally, send an email to name_value@yourdomain.com where name is the part of your email address before the @ in Exchange and yourdomain.com is your domain. The value in “value” will be added to the subject as [value]. You get get an email in this form you know your agent has worked.

Transport Agents and Exchange 2013

I’ve also written an additional blog post on the changes for Exchange Server 2013 and transport agents. This covers the things to consider that are different with regards to Exchange 2007/2010 and the new Exchange.

Creating GeoDNS with Amazon Route 53 DNS

Posted on 3 CommentsPosted in 2013, cloud, exchange, GeoDNS, https, load balancer, mcm, microsoft, MX, networking, owa, smtp, transport

UPDATE: 13 Aug 2014 – Amazon Route 53 now does native GeoDNS within the product – see Amazon Route 53 GeoDNS Routing Policy

A new feature to Exchange 2013 is supported use of a single namespace for your global email infrastructure. For example mail.contoso.com rather than different ones for each region such as uk-mail.contoso.com; usa-mail.contoso.com and apac-mail.contoso.com.
GeoDNS means that you are given the IP address of a server that is in or close to the region that you are in. For example if you work in London and your mailbox is also in London then most of the time you will want to be connected to the London CAS servers as that gives you the best network response. So in Exchange 2010 you would use your local URL of uk-mail.contoso.com and if you used the others you would be told to use uk-mail.contoso.com. For GeoDNS support you use mail.contoso.com and as you are in the UK you get the IP address of the CAS array in London. When you travel to the US (occasionally) you would get the US CAS array IP address, but this CAS array is able to proxy your OWA, RPC/HTTP etc traffic to the UK mailbox servers.
The same is true for email delivery via SMTP. Email that comes from UK sourced IP addresses is on balance a statistical likelihood that it is going to the UK mailbox. So when you look up the MX record for contoso.com from a UK company you get the UK CAS array and the email gets delivered to the CAS array that is in the same site as the target mailbox. If the email is for a user in a different region and it hits the UK CAS array then it is proxied to the other region seamlessly.
GeoDNS is a feature provided by some high-scale DNS providers, but not something Amazon Web Services (AWS) Route 53 provides – so how do I configure GeoDNS with Amazon Web Services (AWS) Route 53 DNS Service?
Quite easily is the answer. Route 53 does not offer GeoDNS but does offer DNS that directs you towards the closest AWS datacentre. If your datacentres are in regions similar to AWS then the DNS redirection that AWS offers is probably accurate.
To set it up, open your Route 53 DNS console, or move your DNS to AWS (it costs $0.50/month for a zone at time of writing, AWS Route 53 pricing here) and then create your global Exchange 2013 namespace record in DNS:

  1. Click Create Record Set and enter the name. In the below example I’m using geo.c7solutions.com as I don’t actually have a globally distributed email infrastructure!
  2. Select A – IPv4 or if you are doing IPv6 select AAAA.
  3. Set Alias to No and enter the IP address of one of your datacentres
  4. Select the AWS region that is closest to this Exchange server(s) and enter a unique description for the Set ID value.
  5. The entry will look something like this:
    image
  6. Save the Record Set and create additional entries for other regions. For the purposes of this blog I have created geo.c7solutions.com in four regions with the following IP addresses:
    Region IP Address Region
    us-east-1 1.2.3.4 Northern Virginia
    us-west-1 6.7.8.9 Northern California
    eu-west-1 2.3.4.5 Ireland
    ap-northwest-1 3.4.5.6 Singapore
    sa-east-1 4.5.6.7 Sao Paulo
    ap-southeast-1 5.6.7.8 Sydney
  7. The configuration in AWS for the remaining entries looks like the following:
    imageimageimage
  8. And also, once created, it appears like this:
    image

In addition to this blog, I’ve left the record described above on my c7solutions.com DNS zone. So depending upon your location in the world, if you open a command prompt and ping geo.c7solutions.com you should get back the IP address for the AWS region closest to you, and so get back an IP that represents a Exchange resource in your global region. Of course the IP’s I have used are not mine to use and probably will not respond to ping requests – but all you need to do is see it DNS returns the IP above that best matches the region that you are in.
I wrote this blog when in a hotel in Orlando and as you can see from the image below, it returns 1.2.3.4 which is the IP address associated with us-east-1.
image
But when I connected to a server in the UK and did the same ping geo.c7solutions.com I got the following, which show GeoDNS working when equating GeoDNS to AWS Latency DNS.
image
What do you get for your regions? Add comments and let us where you are (approximately) and what region you got. If enough people respond from enough places we can see if AWS can go GeoDNS without massive cost.
[Updated 13 Nov 2012] Added Sydney (ap-southeast-1) and fake IP address of 5.6.7.8
[Updated 27 April 2013] Added Northern California (us-west-1) and fake IP of 6.7.8.9

McAfee Total Protection and SMTP Email on Port 587

Posted on Leave a commentPosted in mcafee, smtp

McAfee blocks SMTP connections that do not operate on the default port 25. If you have Outlook, Outlook Express, Windows Mail or Windows Live Mail configured to use an SMTP port that is not 25 then you get errors such as:

Your server has unexpectedly terminated the connection. Possible causes for this include server problems, network problems, or a long period of inactivity. Account: ‘account_name‘, Server: ‘server_name‘, Protocol: SMTP, Port: 587, Secure(SSL): Yes, Error Number: 0x800CCC0F

I understand that this error also occurs in other flavours of McAfee antivirus software, not just the Total Protection variety.

To fix you need to stop McAfee scanning your email before it arrives at the inbox. In McAfee Total Protection for Small Business (TOPS) this is done on the web portal and by visiting Groups + Policies > Policies > Edit Policy for the policy that your client with the problem is using > Advanced Settings and untick Scan email (before delivering to Outlook Inbox).

You need to wait a while (hours) before this setting will take effect, so try again tomorrow and see if the problem has gone away having unticked the option today!