Exchange Edge Server and Common Attachment Blocking In Exchange Online Protection

Posted on Leave a commentPosted in 2007, 2010, 2013, 2016, Edge, EOP, exchange, exchange online, Exchange Online Protection, FOPE, IAmMEC, Office 365

Both Exchange Server Edge role and Exchange Online Protection have an attachment filtering policy. The default in Edge Server is quite long, and the default in EOP is quite short. There is also a few values that are common to both.

So, how do you merge the lists so that your Edge Server attachment filtering policy is copied to Exchange Online in advance of changing your MX record to EOP?

You run

Set-MalwareFilterPolicy Default -FileTypes ade,adp,cpl,app,bas,asx,bat,chm,cmd,com,crt,csh,exe,fxp,hlp,hta,inf,ins,isp,js,jse,ksh,lnk,mda,mdb,mde,mdt,mdw,mdz,msc,msi,msp,mst,ops,pcd,pif,prf,prg,ps1,ps11,ps11xml,ps1xml,ps2,ps2xml,psc1,psc2,reg,scf,scr,sct,shb,shs,url,vb,vbe,vbs,wsc,wsf,wsh,xnk,ace,ani,docm,jar

This takes both the Edge Server default list and the EOP default list, minus the duplicate values and adds them to EOP. If you have a different custom list then use the following PowerShell to get your two lists and then use the above (with “Default” being the name of the policy) PowerShell to update the list in the cloud

Edge Server: Get-AttachmentFilterEntry

EOP: $variable = Get-MalwareFilterPolicy Default
$variable.FileTypes

Exchange Server and Missing Root Certificates

Posted on Leave a commentPosted in 2007, 2010, 2013, exchange, exchange online, Exchange Server, federation, Free/Busy

I came across an issue with a clients Exchange Server deployment today that is not well documented – or rather it is, but you need to know where to look. So I thought I would document the troubleshooting steps and the fix here.

We specifically came across this error when testing Free/Busy for an Office 365 migration, though it could happen for a variety of reasons. Free/Busy and other lookups in a cross-forest Exchange Server deployment require a working organization configuration and this was failing. Running Test-FederationTrust (a prerequisite of the organization relationship) in verbose mode (add -Verbose to the end) returned the following:

Unable to retrieve federation metadata from the security token
service. Reason: Microsoft.Exchange.Management.FederationProvisioning.FederationMetadataException: Unable to access the
Federation Metadata document from the federation partner. Detailed information: “The underlying connection was closed:
Could not establish trust relationship for the SSL/TLS secure channel.”.

The final result of the test will also show two errors for “Unable to retrieve federation metadata from the security token service.” and “Failed to request delegation token.”

The last part of the verbose error is the clue here. The server in question is unable to make an SSL/TLS connection to the endpoint that the federation trust needs to reach to get the federation trust metadata. That endpoint is listed right at the start of the Verbose output. It reads:

VERBOSE: [16:53:08.306 GMT] Test-FederationTrust : Requesting Federation Metadata from
https://nexus.microsoftonline-p.com/FederationMetadata/2006-12/FederationMetadata.xml.

Now that we have a URL and an error message, check that the URL is reachable from each of your Exchange Servers. At my client today we found one server could not successfully reach this endpoint without an SSL error turning up in the browser. The problem was that the certificate that the endpoint is secure with is issued by the Baltimore Cybertrust Root Certificate – one that Microsoft uses for lots of services, but the root certificate was not installed on that machine. Lots of root certs where missing from that machine as it had never had a root certificate update applied to it.

We installed the latest Root Certificate Update and then the federation trust worked and free/busy etc. (mail tips, cross-forest message tracking etc.) all worked fine.

Cannot Send Emails To Office 365 or Exchange Online Protection Using TLS

Posted on 12 CommentsPosted in 2003, 2007, 2010, 2013, exchange, exchange online, Exchange Online Protection, FOPE, hybrid, Office 365, spam, starttls, TLS

I have found this is a common issue. You set up an Exchange Online Hybrid or Exchange Online Protection (EOP) stand alone service and follow all the instructions for the creating of the connectors needed, only to find that your emails queue in your Exchange Server. If you turn on protocol logging you get this error in the log “Connector is configured to send mail only over TLS connections and remote doesn’t support TLS” and if you look at the SMTP protocol verbs that are recorded in the log you see that Microsoft’s servers do not offer STARTTLS as a verb.

STARTTLS is the SMTP verb needed to begin a secure and encrypted session using TLS. Communication between your on-premises servers and Microsoft for hybrid or EOP configurations requires TLS and if you cannot start TLS then your email will queue.

If you are not configuring hybrid or EOP standalone and need to send an email to someone on Office 365 then this is not an issue, because Exchange Server does not require TLS for normal email communication and so the lack of a STARTTLS verb means your email is sent in clear text.

The reason why you are not getting STARTTLS offered is that your connecting IP address is on the Microsoft block list. If you change your connector (temporarily) to allow opportunistic TLS or no TLS at all then your emails will leave the queue – but will be rejected by the Microsoft servers. The NDR for the rejection will tell you to email Microsoft’s delisting service. So now you have an NDR with the answer to the problem in, you can fix it! It takes 1 to 2 hours to get delisted from when Microsoft process your email – so they say it takes 48 hours end to end.

Therefore my recommendation when setting up Exchange Online Hybrid or stand alone EOP is to send an email over plain text to EOP before you configure your service. If you are on the blocklist then you will get back the delisting email and you can process that whilst setting up the connectors to Office 365 and so by the time you are ready to test, you are off the blocklist!

To send a test email over Telnet

  1. Install the Telnet Client feature on your Exchange Server that will be your source server for hybrid or connectivity to EOP for outbound email
  2. Type the following. This will send an email to a fake address at Microsoft, but will hit the TLS error before the message is rejected

    telnet microsoft-com.mail.protection.outlook.com 25

  3. You are now connected to Exchange Online Protection and you should get a 220 response
  4. Type the following to send the email by command line. No typo’s allowed in telnet, so type carefully. Replacing your email address where prompted so that you get the NDR back to you.

    ehlo yourdomainname.com
    mail from: youremailaddress@domainname.com
    rcpt to: madeupaddress@microsoft.com
    data
    from: Your Name <youremailaddress@domainname.com>
    to: madeupaddress@microsoft.com
    subject: testing to see if my IP is blocked

    type something here, it does not matter what, this is the body of the message you are sending
    .

  5. A few points about the above. It must finish with a . (full stop) on a line by itself followed by a carriage return. There must be a blank line between the subject line and the body. And finally, for each line of data you type, the Microsoft SMTP servers will return either a 250 or 354 response.

Ensuring Email Delivery Security with Exchange 2013

Posted on 4 CommentsPosted in 2007, 2010, 2013, encryption, exchange, IAmMEC, TLS, transport

To force Exchange 2013 to guarantee the secure delivery of a message can be done a few different ways. In this version of the product and in previous versions it was possible to create a send connector for a given domain and enable Mutual TLS on the connector. Then all messages to the domain(s) that this connector serviced would need to travel over a TLS connection where the certificate at both ends was completely valid (i.e. valid regards the date, had the correct subject or SAN for the domain, was issued by a trusted certificate authority etc.). In previous versions (2007 to 2010 again) it was possible to enable Domain Secure and add another level of checks to the Mutual TLS session. Domain Secure does not work in Exchange 2013.

And great though these methods of transport security are, they are limited in that they are difficult to set up (require good knowledge of certificates) and needs to be properly configured at both ends of the connection. They are also limited in that they will only secure email to the selected domains. If you need to send a “top secret” email to someone, you don’t really want to have to configure a connector at both ends and force all email for that domain down the same path.

So, in Exchange 2013 you can create a transport rule to force the connection to use TLS, and if TLS fails then have the message queue on the sender until it retries and eventually expires. If TLS is never available, the message never goes out of Exchange – or so it would be if all you read was the description in the documentation!

The RouteMessageOutboundRequireTLS transport rule action (or the Secure the message with > TLS encryption option in the ECP transport rules wizard) allows you to craft a rule for any condition (for example the subject or body contains any of these words: top secret) which will require the email to use an encrypted session for the delivery outside of Exchange Server. Note that for this to work the TLS session does not need to be protected by a given certificate or valid etc., it just needs the receiving SMTP Server to offer STARTTLS and for the encryption to work.

And it needs a source server for sending the message in every Active Directory site within the organization. Currently (as of CU2 for Exchange 2013) if you send a message from a site that does not contain a send connector that can handle the message to the internet then Exchange will pass it to a site that can, but the source transport server will now not enforce the TLS requirement and will send the message unprotected if STARTTLS is not offered.

So if you want to guarantee the use of TLS for certain types of message use the RouteMessageOutboundRequireTLS transport rule condition and ensure that you do not need to do cross site delivery of messages to reach a send connector source server to delivery the message to the internet.

Moving Exchange Online Protection Junk Mail to the Junk Email Folder

Posted on Leave a commentPosted in 2007, 2010, 2013, active directory, cloud, Edge, EOP, exchange, exchange online, FOPE, mcm, mcsm, spam, transport

If you use Exchange Online Protection (EOP) to filter your email in the cloud and to remove spam and malware before onward delivery to you, and if you use Exchange 2007 or later on-premises, then you need to configure Exchange to move detected spam to the Junk Email folder in Outlook.

By default EOP detects two levels of spam (malware is automatically removed) and tags them. In Exchange, you need to use Transport Rules to move these emails to the users Junk Email folder. If you wish for EOP throw away junk emails of either detection level (i.e. high junk is discarded) then you still need to configure these transport rules to move the remaining detected junk email into the Junk Email folder in Outlook. If you place all EOP detected spam in the quarantine or throw it away, you should still create these transport rules as it means they are in place for any future changes you make at EOP.

If you use Exchange Online (part of Office 365) then you do not need to create these rules as they already exist (though you cannot see them in any admin tools you have).

The Transport Rules To Create

The following four transport rules need to be created on your Exchange organization. These four rules get created with the highest priority, moving all existing rules below them. They set the Spam Confidence Level of the message to 6 in these examples, though this should be set to a value that exceeds the SCLJunkThreshold organization wide setting for your Exchange Organization, as any email that exceeds this value is placed into the Junk Email folder upon delivery to the users mailbox.

New-TransportRule "Move EOP Detected Spam (SFV:SPM) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SFV:SPM" -SetSCL 6 -Priority 0

New-TransportRule "Move EOP Detected Spam (SFV:SKS) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SFV:SKS" -SetSCL 6 -Priority 1

New-TransportRule "Move Personal Block List Spam (SFV:SKB) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SFV:SKB" -SetSCL 6 -Priority 2

New-TransportRule "Move Bulk Email (SFV:SKB) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SRV:BULK" -SetSCL 6 -Priority 3

Once these four rules are run, AD replication will need to be allowed to happen and then all Exchange Servers will action this rule for any email that EOP thinks is spam. The default value for SCLJunkThreshold is 4, so as long as the rules set the SCL value to greater than this value it should work. Use Get-OrganizationConfig | FL SCLJunkThreshold to get this value for your organization.

What the EOP Message Tags Mean

The X-Forefront-Antispam-Report header in the email contains many values. See http://technet.microsoft.com/en-us/library/dn205071(v=exchg.150).aspx for what EOP thinks about the spam rating of the message.

  • SFV – means Spam Filtering Verdict
  • SFV:SFE – Originated from a Safe Sender (EOP learns Outlook safe senders due to Windows Azure Directory Sync)
  • SFV:BLK – Originated from a Blocked Sender
  • SFV:SPM – Spam
  • SFV:SKB – Emails where the sender is on the users personal block list as configured in OWA or Outlook Junk Email settings
  • SFV:SKS – (SKIP) The message was marked as spam prior to being processed by the spam filter. This includes messages where the message matched a Transport rule to automatically mark it as spam and bypass all additional filtering
  • SFV:NSPM – Not Spam

Article updated in June 2016 to include new headers and go from two rules to four

Removing Edge Subscription When Exchange 2013 Installed

Posted on 6 CommentsPosted in 2007, 2010, 2013, Edge, exchange, federation, IAmMEC, mcm, mcsm, smarthost, transport

Exchange 2013 does not have an Edge role (at the time of writing – Aug 2013). It is possible to use Exchange 2010 SP3 and install the Edge role should you need one.

There is a problem though when it comes to removing the Edge Subscription between an organization that contains Exchange 2013 servers and the Exchange 2010 Edge Server. To remove the subscription on the Edge server role you run Remove-EdgeSubscription servername and this removes both the subscription and any subscribed objects from the local AD LDS database on that Edge Server. But if any of these subscribed objects where created on Exchange 2013 after it was installed, then they will have an ExchangeVersion equal to 0.20 (15.0.0.0). The Exchange 2010 SP3 Remove-EdgeSubscription cannot process this object and so fails with:

Remove-EdgeSubscription : Can’t make this change because the object’s ExchangeVersion property is 0.20 (15.0.0.0), which is not supported by the current version 0.1 (8.0.535.0). You will need a newer version of Exchange to make this change. Property Name: ExchangeVersion
At line:1 char:24
+ Remove-EdgeSubscription <<<<  edge2 -Verbose
+ CategoryInfo          : NotSpecified: (0:Int32) [Remove-EdgeSubscription], DataValidationException
+ FullyQualifiedErrorId : D8A49A14,Microsoft.Exchange.Management.SystemConfigurationTasks.RemoveEdgeSubscription

The way to fix this is to find and manually remove the object with an ExchangeVersion of 0.20 (15.0.0.0) from the AD LDS database and then repeat the Remove-EdgeSubscription cmdlet – as that should now work (unless you have two or more objects with the higher version number to locate and delete).

  1. To find objects with an ExchangeVersion greater than “0.1 (8.0.535.0)”, which is the version Exchange 2010 will process, open ADSIEdit on the Edge server.
  2. Right-click the ADSI Edit node at the top of the window and choose Connect to…
    image
  3. In the Connection Settings dialog (shown above), change the Select a well known Naming Context to Configuration and type the local server name and the AD LDS port in the Select or type a domain or server field. The server:port value should be EDGESERVERNAME:50389
    image
  4. Expand the tree view until you reach CN=First Organization,CN=Microsoft Exchange,CN=Services,CN=Configuration,CN={AD258B4D-CCB4-4125-80C1-7B73CE066341}
    image
  5. You now need to look at each object, starting with those you remember having created since 2013 was installed, for an object who’s msExchVersion value is not 4535486012416. For example, in the below screenshot I have an accepted domain made due to Federation in Exchange 2013. This object (under CN=Accepted Domains,CN=Transport Settings,…) has a value of 88218628259840.
    image
  6. To validate that this is the correct object to manually delete, from the Exchange Management Shell on the Edge server, enter Get-Object | FT name,ExchangeVersion where Object is the cmdlet that you are looking to query – in my case it would be Get-AcceptedDomain.
    image
  7. As you can see, this object has a newer ExchangeVersion and so it is (at least) this object that is stopping Exchange EdgeSync from being removed.
  8. Manually delete this object in ADSI Edit (it is safe to do this as it will resync from the Exchange organization if you recreate Edge Subscription later). Do not delete it from the Active Directory with ADSI Edit – just from AD LDS. Take care to only delete this object and not the parent object.
  9. Once this object is gone, try Remove-EdgeSubscription servername again. If this is the only object, then the Edge Subscription will be removed in the Edge Server

You can now carry on with whatever it was that you were doing that required a removal of the Edge Subscription.

Domain Secure and Edge Servers

Posted on Leave a commentPosted in 2007, 2010, 2013, certificates, cloud, exchange, firewall, smarthost, smtp, transport

I was asked a question recently on the Microsoft Certified Master course for Exchange 2010 and was told that the answer was not clearly written up on the internet. So I thought I would write this blog post. The question was based on the idea that Domain Secure worked from a Hub Transport server in the classroom lab but not when mail flow went via an Edge server.

Domain Secure is end to end security, it cannot have anything in the middle – i.e. it cannot go via an Exchange Edge server, an Exchange 2013 Frontend Server or a third party SMTP relay.

The SMTP client in the connection (the send connector host) needs to connect to the SMTP server (the receive connector host) and swap certificates and prove the other side is who the other side say they are – i.e. mutual authentication. Also the domains must match the TLS list in TransportConfig (TLSSendDomainSecureList and TLSReceiveDomainSecureList). Therefore anything in the middle will offer a different certificate and so Domain Secure fails.

If there is a middle party and you want to do mutual authentication (i.e. swap certs to prove who you are), with one party offering their cert and not the cert of the final recipient domain (i.e. mail.messaging.microsoft.com or postini.com etc.) then use TLSAuthLevel and the DomainValidation option on the send connector (an SP1 addition to Exchange 2010). No green ticky ticky though.

Edge can do Domain Secure though. But Edge needs to be the starting point, i.e. the host of the send connector. So configure Domain Secure on the Edge (i.e. set the certificates and correct firewall settings) and ensure that the send connector for Domain Secure has the Edge server as the source. Ensure Edge Domain Secure receive connector is the target for inbound as well if you want it to work both ways. And of course you need working EdgeSync so hubs can deliver to Edge so that Edge can deliver emails for you.

Forefront Online Protection for Exchange Spam Filtering to Outlook’s Junk E-Mail Folder

Posted on Leave a commentPosted in 2007, 2010, 2013, cloud, dirsync, EOP, exchange, exchange online, FOPE, hosting

Forefront Online Protection for Exchange (FOPE) is a cloud hosted email anti-spam and antivirus filtering system. Amongst the options to filter away your spam, one of the options to to allow the email to be flagged and sent on into your on-premises email system, and then managing it there.

If you have Exchange 2007 or later it is possible to write Exchange Transport Rules to process this flagged email and move it directly to the Junk E-Mail folder in your mailbox. This allows users to have their probable spam in a different location from their inbox email, but not in a different system accessed external to their email client, for which they might need a second login account or a delay before receiving the notification email. This works for probable spam as much obvious spam is filtered out at the edge of FOPE and so cannot make it to a place where users can see it.

An additional benefit of this filtering inside Outlook or OWA to the Junk E-Mail folder is that users can mark messages as safe or blocked in the client and this is picked up by Exchange and can be sent automatically to FOPE, which means FOPE flags it as spam before it reaches the Exchange organization.

To configure this you need to set FOPE to flag spam with a X-Header. This is documented at various places online, but misses out one vital piece of information which I wrote this blog to document. The missing info is what the value of the X-Header is so that you can actually write a transport rule to process it.

In FOPE, select your email domain (under Domains) and on the domain page click Edit next to Spam Action under Service Settings. Change the Spam Action to Add X-Header and type the header name that you want to use:

image

On your Exchange organization create a transport rule (these pictures are from Exchange 2007, but 2010 or 2013 are technically the same though visibly different). The transport rule is set to apply to messages when a message header contains specific words and the name of the header is the value set in FOPE previously (X-MoveTo-JunkEmail in this example). The value of the header will always read “This message appears to be spam.”.

image

It is possible to use the when the message header contains text patterns and use the RegEx expression \w* to find emails with your header in it and containing a value (\w* means any letter or digit repeated), but as we know the value for the header is always “This message appears to be spam.” then using regular expression filtering is adding un-needed CPU cycles to the Exchange Server – only use RegEx when the value can vary.

Office 365 use this process to place your probable spam in Junk E-Mail. In their case the header is X-FOSE-spam.

The transport rule continues to set the spam confidence level to a high value of your choosing, and higher than the value we will set in OrganizationConfig below.

So this rule will take all emails the FOPE marks as spam and changes the spam confidence level (SCL) value to 9 (in this example). Finally we need to set the SCLJunkThreshold property of OrganizationConfig to a value below the value in the transport rule. Exchange will place all email that exceeds this threshold into the Junk E-Mail folder in Outlook:

Set-OrganizationConfig -SCLJunkThreshold 4

If you are running the Content Filter hygiene agent then you will also want to check the Get-ContentFilterConfig values for SCLRejectEnabled, SCLDeleteEnabled and SCLQuarantineEnabled are all set to false. This ensures that SCL values that are are high are not rejected or deleted, or sent to quarantine. As all your email should be filtered by FOPE if you are using it, and the firewalls at your company or receive connectors on Exchange should be blocking email not sent from the FOPE datacenters (in FOPE admin pages click Information tab and then the Configuration link to get the list of IP addresses). The content filtering agent can be used as a second filter on-premises but if you don’t want to throw away or reject spam at this second level (recommended in this scenario) then ensure that the filter rejection, delete and quarantine settings are disabled. If you want to delete probable spam then set the transport rule to 5 and the SCLDeleteEnabled to $true and the SCLDeleteThreshold value to 9. Don’t reject or use the on-premises quarantine features when using FOPE (the transport rule cannot process the quarantined messages for a start).

Finally for administrators, consider a Message Retention Management or Retention Policy to delete without recovery email in Junk E-Mail folder after 21 days. Also consider the FOPE Directory Sync tool to push the user lists to FOPE as this upload also includes the pushing of the safe senders information as well.

Now for your users, all probable spam is managed in their email client, integrated with safe sender lists and without resorting to another application to view and deliver false positives and spam they want to read!

Exchange 2013 Transport Agents

Posted on 3 CommentsPosted in 2007, 2010, 2013, agent, exchange, sdk, smtp, visual studio

Earlier today I posted http://blog.c7solutions.com/2012/10/creating-simple-exchange-server.html on how to create a transport agent in Exchange, and though the steps cover some of the detail for Exchange Server 2013 they do not cover some of the detail, so I’ve added that to this post below:

  • Use .NET Framework 4.0 for Exchange 2013 though it is possible to write the agents using earlier versions of the .NET Framework. If you do use earlier versions then you need to update some of the config files on the server to state the version levels that you support and Enable Support for Legacy Transport Agents on Technet lists the steps for this.
  • The Front End Transport role (which is installed on the CAS Server role) can support SMTP agents bound to events up to, but not including OnEndOfData.
  • The Hub Transport role (which is installed on the Mailbox Server role) can support all SMTP agents except for those that bind to the OnConnect event.
  • The Hub Transport role also support routing agents and delivery agents. The CAS role supports neither of these agent types.
  • The Exchange 2013 Edge Transport Server role (expected with Exchange 2013 SP1) will support agents at the OnConnect event.
  • To install a transport agent on a multi-role server (CAS and Mailbox on the same machine) then you need to use can use -TransportService Hub to the Install-TransportAgent cmdlet shown in the earlier blog if you want to bind the agent to the Hub Transport events. If you want to use the Front End Transport events (OnMailFrom to OnEndOfHeaders) then use –TransportService FrontEnd instead (updated 2 Oct from Philippe comment below).
  • To install a transport agent on a CAS only server role you need to use local PowerShell and load the Exchange Management Shell snap-in with Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010
  • Finally, to view the agents in a given service, always state the service name or you will get back the Hub Transport service by default. For example Get-TransportAgent –TransportService FrontEnd for the agents bound to the Front End Transport service. Other values are Hub, Edge and MailboxSubmission and MailboxDelivery (though the last two don’t seem to work in the pre-release version on Exchange Server 2013.

Creating a Simple Exchange Server Transport Agent

Posted on 5 CommentsPosted in 2007, 2010, 2013, agent, exchange, mcm, sdk, smtp, transport agent, visual studio

This blog post follows a session that I delivered at the MEC 2012 conference in Orlando. If you attended the conference the slides are available on http://mymec.mecisback.com for the rest of 2012.

Part of the transport agents session was writing a new transport agent, and the example agent was to do add a form of catch-all functionality. The example code below (which purposefully needs editing to work, as its designed to be a learning tool) takes an email in the form of alias_tag@domain.com where the final recipient is alias@domain.com and the subject line is changed to read [tag] Original Subject. Therefore email addresses can be given out that are an adjusted form of your real address, and on receipt of the email the correct recipient address is determined in code (the value before the _ appended to the domain name) so that you do not need to add lots of aliases to your user account in Active Directory. The subject is then changed to indicate the email came from this alternative original address. For example, an email to bill_linkedin@microsoft.com would go to bill@microsoft.com and have [linkedin] added to the start of the subject.

As this partial catch all transport agent needs access to the recipient information and the subject, the agent needs to be bound to OnEndOfHeaders or OnEndOfData in the SMTP stack. It could also be bound to OnSubmittedMessage as a routing agent (rather than an SMTP agent) but could not be bound to OnResolvedMessage as recipient resolution has already happened by this point.

Writing the Transport Agent

To create the transport agent you need Visual Studio – the basic version of Visual Studio are sufficient and you need to copy the two DLL’s from Program Files\Microsoft\Exchange Server\V14\Public (or the V15 folder for Exchange 2013) to a folder on your development machine. You need a copy of these DLL’s for every version/service pack/update rollup of Exchange that you will run your agent on, as the DLL’s might change between these versions, and if you have the wrong version you will not be able to install your agent on a new server or if the server is updated, the transport service will fail as it will not be able to load the agent.

In Visual Studio, create a .NET 2 or 3.5 Class Library (for Visual Basic if using the below code) for Exchange 2010 or .NET 4.0 Class Library for Exchange 2013. For the project name enter something descriptive for what you are going to create (rather then ClassLibrary1). For example MECDemoPartialCatchAll.

image

Copy and paste the following code into the class.vb file, replacing the template text.

   1: Imports System

   2: Imports System.Collections.Generic

   3: Imports System.Text

   4: Imports Microsoft.Exchange.Data.Transport

   5: Imports Microsoft.Exchange.Data.Transport.Smtp

   6:

   7: Namespace XXXXX REM Change This

   8:

   9:     NotInheritable Class YYYYYY REM Change This

  10:         Inherits SmtpReceiveAgentFactory

  11:

  12:         Public Overrides Function CreateAgent(ByVal server As SmtpServer) As SmtpReceiveAgent

  13:             Return New ZZZZZ REM Change This

  14:         End Function

  15:

  16:     End Class

  17:

  18:     Public Class ZZZZZ REM Change This

  19:         Inherits SmtpReceiveAgent

  20:

  21:         Private Sub MyEndOfDataHandler(ByVal source As ReceiveMessageEventSource, ByVal e As EndOfDataEventArgs) Handles Me REM Change This

  22:             ' Get and change the recipient from alias_tag to alias (only doing for 1 recipient for simplicity)

  23:             If e.MailItem.Recipients.Count = 1 And InStr(e.MailItem.Recipients.Item(0).Address, "_") > 1 Then

  24:                 Dim Recipient() As String = Split(e.MailItem.Recipients.Item(0).Address, "@", 2)

  25:                 Dim EmailAlias() As String = Split(Recipient(0), "_", 2)

  26:                 'EmailAlias(0) = alias to send email to

  27:                 'EmailAlias(1) = tag for subject line

  28:                 'Recipient(1) = domain

  29:

  30:                 ' The following line prepends [tag] to the subject of the message.

  31: REM Change This                 "[" + EmailAlias(1).ToString + "] " + e.MailItem.Message.Subject

  32:

  33:                 'the following drops the current recipient

  34:                 e.MailItem.Recipients.Remove(e.MailItem.Recipients.Item(0).Address)

  35:

  36:                 'the following adds the recipient back again, this time using the alias without the tag

  37:                 e.MailItem.Recipients.Add(New RoutingAddress(EmailAlias(0).ToString + "@" + Recipient(1).ToString))

  38:             End If

  39:         End Sub

  40:

  41:     End Class

  42:

  43: End Namespace

Download this code

The remaining steps on creating the agent will be to modify the lines above that are marked with REM statements and then to add the reference DLL’s you copied from your Exchange Server and finally build your DLL.

Visual Studio can tell you when your code contains errors before you build your code, but to do so you need to reference the DLL’s that you obtained from your Exchange Server earlier in this blog. To do this you need to have copied the two DLL’s to a unique folder on your computer. I use c:\temp\Agent Authoring\Dll\E14-SP2-RU4 to store the DLL’s from Exchange 2010, SP2, RU4. Then when RU5 is released, if the DLL’s have changed I will place them in an E14-SP2-RU5 folder and update the references in my project. If I keep using the same folder then Visual Studio does not refresh the DLL but uses an existing cached copy.

To add the DLL’s right-click the project name to the right of the Visual Studio window and choose Properties (or press Alt+Enter):

image

Change to the Reference’s tab and click Add > Browse to find and add these two DLL’s.

image

You should see the two DLL’s listed as well as the default references for DLL’s included in the class library template that you used in Visual Studio.

Back on the code, its time to make the changes. Each of the changes and why it is needed are detailed based on the line numbers above

Line 7 – Namespace

This value of the namespace for the transport agent. Standard conventions indicate that this should be your company name. Therefore in your code change XXXXX to C7Solutions

Line 9 – NotInheritable Class

This value, YYYYYY, is the class that you are creating. This class inherits all the functionality of the Microsoft.Exchange.Data.Transport.Smtp.SmtpReceiveAgentFactory class. For this partial catch-all agent, a good name would be PartialCatchAllFactory.

Line 13 and 18 – Public Class

The Public Class contains the code to execute when the agent is called. This class has a name (ZZZZZ in the sample code above) and should be given a name that represents what the code does. This name in this example can be PartialCatchAll and it needs to be used in Line 13 (inside the PartialCatchAllFactory code to indicate the code to call instead of SmtpReceiveAgentFactory. And of course it is needed on line 18 to name the actual block of code. The name to use for this example will be PartialCatchAll

Line 21 – Handler

This line is missing the end of the code. Its a line that says run the code in this subroutine if the OnEndOfData event is called. Change the end of the line to read Me.OnEndOfData. This value should be one of the suggestions in the drop down list if you have done everything correct so far:

image

This subroutine also returns e as a pointer to the email message (i.e. you can modify e.MailItem.DeliveryPriority and many other properties) and source as a reference to the SMTP connection (i.e. with source.Disconnect you would close the SMTP session).

In this code lines 34 and 37 change the recipient. Line 34 removes the first recipient and then line 37 adds a new recipient that matches the alias + the domain.

Line 31 – Modify Subject

Finally, this code is missing the start of line 31. You need to enter e.MailItem.Message.Subject =  at the start of the line so that they subject becomes [tag] + original subject.

Building the Transport Agent DLL

The Errors List at the bottom of the Visual Studio screen should be empty by now. So time to build the project. If you have not saved this project in Visual Studio, the DLL will be created in the %temp% directory, so best recommendation is to save the project before building the DLL.

To save the project click File >  Save All and enter a suitable name for the project.

image

The project name becomes the solution name by default and then click Save.

The final thing to check before you build the DLL is the Root namespace value. This should be set to the name of the class library and can be set during creation of the library. If you have written your library in VB.Net then you will need this value to register the DLL. If you used C# then you will not need this value. As we have used VB.Net above, we need to check the root namespace value. This can be found on the project properties page.

image

If your DLL is ready to release (i.e. you have build and tested it already) then choose Build > Configuration Manager menu. Change the Active solution configuration to Release and click Close. If you do not make this change then the DLL will be created in the project_name\bin\debug folder and its possible that Visual Studio does not compile all the optimizations to the code that it can. For production items that perform at as good a speed as you can write optimal code, you should change this to Release for the Configuration value. The DLL will be created in project_folder\bin\release.

To build your DLL (debug or release) select the Build > Build RootNamespaceValue menu. This will create a DLL in project_name\bin\debug or project_name\bin\release.  A working version for Exchange 2010 SP2, RU4 can be downloaded from here.

Installing the Transport Agent

Copy the DLL to all your Exchange Servers that have the hub transport role installed on them (or if this should only apply to emails inbound from the internet, then just the servers that are listed on your MX records or the first servers connected to for inbound emails). Place the file in a directory (say C:\C7Agents) and run the following five lines from Exchange Management Shell:

Install-TransportAgent -Name “C7PartialCatchAll” -AssemblyPath “C:\C7Agents\MECDemoPartialCatchAll.dll”
-TransportAgentFactory “MECDemoPartialCatchAll.C7Solutions.PartialCatchAllFactory”

Enable-TransportAgent “C7CatchAll”

Restart-Service MSExchangeTransport

IISRESET

# Recommend closing EMS window now

Note that this restarts the transport service (required) and IIS (which will effect OWA and other CAS roles on the same machine). Remote Powershell (an IIS resource) will lock the DLL open, and so if you need to delete the DLL after uninstalling it, you need to have reset IIS and closed the Powershell window.

Finally, send an email to name_value@yourdomain.com where name is the part of your email address before the @ in Exchange and yourdomain.com is your domain. The value in “value” will be added to the subject as [value]. You get get an email in this form you know your agent has worked.

Transport Agents and Exchange 2013

I’ve also written an additional blog post on the changes for Exchange Server 2013 and transport agents. This covers the things to consider that are different with regards to Exchange 2007/2010 and the new Exchange.

Installing and Configuring AD RMS and Exchange Server

Posted on 4 CommentsPosted in 2007, 2008 R2, 2010, active directory, certificates, exchange, exchange online, microsoft, networking, Office 365, organization relationships, owa, rms, server administrator

Earlier this week at the Microsoft Exchange Conference (MEC 2012) I led a session titled Configuring Rights Management Server for Office 365 and Exchange On-Premises [E14.314]. This blog shows three videos covering installation, configuration and integration of RMS with Exchange 2010 and Office 365. For Exchange 2013, the steps are mostly identical.

Installing AD RMS

This video looks at the steps to install AD RMS. For the purposes of the demonstration, this is a single server lab deployment running Windows Server 2008 R2, Exchange Server 2010 (Mailbox, CAS and Hub roles) and is the domain controller for the domain. As it is a domain controller, a few of the install steps are slightly different (those that are to do with user accounts) and these changes are pointed out in the video, as the recommendation is to install AD RMS on its own server or set of servers behind a IP load balancer.

Configuring AD RMS for Exchange 2010

The second video looks at the configuration of AD RMS for use in Exchange. For the purposes of the demonstration, this is a single server lab deployment running Windows Server 2008 R2, Exchange Server 2010 (Mailbox, CAS and Hub roles) and is the domain controller for the domain. This video looks at the default ‘Do Not Forward’ restriction as well as creating new templates for use in Exchange Server (OWA and Transport Rules) and then publishing these templates so they can be used in Outlook and other Microsoft Office products.

 

Integrating AD RMS with Office 365

The third video looks at the steps needed to ensure that your Office 365 mailboxes can use the RMS server on premises. The steps include exporting and importing the Trusted Publishing Domain (the TPD) and then marking the templates as distributed (i.e. available for use). The video finishes with a demo of the templates in action.

Access Is Denied Message After Sysprep–How To Fix

Posted on 1 CommentPosted in 2003, 2007, 2008, 2008 R2, 2012, 64 bit, backup, bios, hyper-v, password, recovery, sysprep, windows, windows 2003, windows 2008, windows 7, windows server, workstation, x64, x86

If before you use Sysprep to prepare a Windows machine for imaging you set the administrators password “User cannot change password” then sysprep will not clear this setting, but will set the “User must change password at next logon” setting. Normally these two settings are mutually exclusive, but in the scenario for sysprep it seems they can both end up being set.

This means you get prompted to reset you password at first logon after sysprep completes and then find you have “Access Denied” as the response. There is seemingly no way around this Catch-22.

That is unless you use the Offline NT Password and Registry Editor. This tool allows password resets when booting the server from a CD or USB key (so physical access to the server is required). As the download for this is an iso file, it can also be used in virtual environments by configuring your virtual machine to boot from the iso you have downloaded.

To allow you to logon to your machine following the above issue, all you need to in the Offline NT Password tool is to blank out the administrators password and unlock the account. These are options 1 and 4 during the password reset stage. Full instructions with screenshots follow:

  1. Boot the server with the issue with the Offline NT Password and Registry Editor iso file:
    image
  2. Choose the correct boot option (or just press Enter for the defaults):
    image
  3. For Vista and earlier select the default of Option 1. For Windows 7 and Windows 2008 and later select Option 2 (to boot into the second partition on the disk). You might need to select a different option if you have more partitions. You need to select the partition that Windows is installed on.
  4. If the disk is marked as Read-Only ensure that the server went through a clean boot and was not shutdown incorrectly. Once the messages indicate a writable partition
    image
  5. Select the presented folder (by pressing Enter again). You can typically just press Enter through most of these stages. You will be asked what you want to do – we want to reset passwords:
    image
  6. Select Option 1 to Edit user data and passwords:
    image
  7. Press Enter to choose the Administrator account:
    image
  8. Type 1 to Clear (blank) user password. You should get back the message “Password cleared!”:
    image
  9. Press Enter again to reselect the Administrator account, and this time select Option 4 to unlock the account (even though this program tells you the account is already unlocked):
    image
  10. Once you see “Unlocked!” you can quit from this program. The process to quit requires you to save your changes. Note that the default setting is not to save changes, so you cannot now use Enter to select the default option.
  11. Enter ! to quit from the password reset program:
    image
  12. Enter q to quit from the script and to ask about saving changes:
    image
  13. Enter y to write back the files that have been changed:
    image
  14. You should have been told “***** EDIT COMPLETE *****”. Press Enter to finish the program scripts:
    image
  15. At this final screen you can remove the CD or unmount the iso image from your virtual machine and press CTRL+ALT+DEL to restart the server. The server should now boot into Windows and auto-logon as it has a blank password.
  16. Change the password and optionally untick the “User cannot change password” setting.

How To Speed Up Exchange Server Transport Logging

Posted on 1 CommentPosted in 2007, 2010, 2013, exchange, mcm, microsoft, transport

In Exchange 2010 SP1 and later any writing to the transport log files for activity logging (not the transaction logging on the mail.que database) is cached in RAM and written to disk every five minutes.

In a lab environment you might be impacted by this as you might have sent an email and want to check the logs for the information they contain for diagnostic reasons. The problem is you might need to wait five minutes to get this information.

Exchange 2010

To reduce the memory cache time to 30 seconds set the following two entries in the Edge.Transport.exe.config file (found in \Program Files\Microsoft\Exchange Server\v14\bin) within the AppSettings area:

<add key="SmtpSendLogFlushInterval" value="0:00:30" />
<add key="SmtpRecvLogFlushInterval" value="0:00:30" />

The two values above control different log files. Each transport log file has a different setting – so its possible to set Receive Connector protocol logging to a different value from Send Connector protocol logging if you wanted to. Once you make your changes to Edge.Transport.exe.config you need to restart the Microsoft Exchange Transport service for the changes to be picked up.

Here is a list of the properties that I know about that can be changed:

  • SmtpSendLogFlushInterval – Timespan value on how often to write the Send Connector protocol logging log to disk
  • SmtpRecvLogFlushInterval – Timespan value on how often to write the Receive Connector protocol logging log to disk
  • ConnectivityLogFlushInterval – Timespan value on how often the Connectivity log is written to disk.

In addition to the above, which are all timespan values for how often to write to disk, if the memory buffer that contains the log entries fills up then it will be written to disk as well. The default memory buffers are 1MB. So on a very busy server you might find that the log writing is not every five minutes exactly but of a more “random” nature as the buffer is filled. The following settings control the size of the buffer for the above timespans:

  • SmtpSendLogBufferSize
  • SmtpRecvLogBufferSize
  • ConnectivityLogBufferSize

Exchange 2013 CU1 and Later

The process for this version of Exchange is similar, just different log files because of the different services in use.

In Exchange 2013 you have transport services for mailbox (submission and delivery), transport core and CAS (frontend transport). The config files for these services are:

  • EdgeTransport.exe.config (transport core)
  • MSExchangeFrontEndTransport.exe.config (Frontend Transport on the CAS role)
  • MSExchangeDelivery.exe.config (for mailbox delivery on the Mailbox role)
  • MSExchangeSubmission.exe.config (for mailbox submission on the Mailbox role)

You will find these config files in \Program Files\Microsoft\Exchange Server\v15\bin.

 

Highly Available Geo Redundancy with Outbound Send Connectors in Exchange 2003 and Later

Posted on 6 CommentsPosted in 2003, 2007, 2010, cloud, DNS, domain, door, exchange, exchange online, load balancer, loadbalancer, mcm, microsoft, MX, Office 365, smarthost

This is something I’ve been meaning to write down for a while. I wrote an answer for this question to LinkedIn about a week ago and I’ve just emailed a MCM Exchange consultant with this – so here we go…

If you configure a Send Connector (Exchange 2007 and 2010) or Exchange 2003 SMTP Connector with multiple smarthosts for delivery to, then Exchange will round-robin across them all equally. This gives high availability, as if a smarthost is unavailable then Exchange will pick the next one and mail will get delivered, but it does not give redundancy across sites. If you add a smarthost in a remote site to the send connector Exchange will use it in turn equally.

So how can get get geographical redundancy with outbound smarthosts? Quite easily it appears, and it all uses a feature of Exchange that’s been around for a while. But first these important points:

  • This works for smarthost delivery and not MX (i.e. DNS) delivery.
  • This is only useful for companies with multiple sites, internet connections in these sites and smarthosts in those sites.
  • This is typically done on your internet send connectors, the ones using the * address space.

You do this by creating a fake domain in DNS. Lets say smarthost.local and then creating A records in this zone for each SMTP smarthost (i.e. mail.oxford.smarthost.local). Then create an MX record for your first site (oxford.smarthost.local MX 10 mail.oxford.smarthost.local). Repeat for each site, where oxford is the site name of the first site in this example.

Then you create second MX records, lower priority, in any site but use the A record of a smarthost in a different site (oxford.smarthost.local MX 20 mail.cambridge.smarthost.local).

Then add oxford.smarthost.local as the target smarthost in the send connector. Exchange will look up the address in DNS as MX first, A record second, IP address last), so it will find the MX record and resolve the A records for the highest priority for the domain and then round-robin across these A records.

If you have more than one smarthost in a site, add more than one MX 10 record, one per smarthost. Exchange will round-robin across the 10’s. When all the 10’s are offline then Exchange will automatically route to mail.cambridge.smarthost.local (MX priority 20 for the oxford site) without needing to disable the connector and retry the queues.

If you used servernames and not MX’s then it would round-robin amongst all entries, and so equally sent email to Cambridge for delivery. The MX option keeps mail in site for delivery until it cannot and then sends it automatically to the failover site.

Restricting Message Sizes in Exchange Server to Low Bandwidth Sites

Posted on Leave a commentPosted in 2007, 2010, exchange, mcm, windows 2008

Exchange Server has a series of different settings for controlling the maximum message size into and around an Exchange organization, but what about when parts of your organization have a considerably lower bandwidth than other parts, for example offices with servers in rural or hard to reach locations and require satellite WAN links or ships that are at sea.

For these and other examples it has been possible to limit the message size sent and from these limited bandwidth sites since Exchange Server 2007 SP1 by setting the MaxMessageSize property in Set-AdSiteLink

Set-AdSiteLink TitanicSiteLink -MaxMessageSize 2MB

Once an email is sent to a recipient in the target site Exchange Server (as part of the Categorizer component) determines the least cost route and sends the email. If the least cost route includes the site link on which you have limited your bandwidth then the email will be returned to the sender as an NDR if it exceeds the MaxMessageSize limit. If you only have one AD Site Link to your linited bandwidth site then Exchange routing will have to use that link. If you have more than one AD Site Link make sure they are all set to the limited size to that whatever the calculated least cost route is, the size limit will be enforced.

The only problem with this is that Exchange does not have the correct permissions within the Active Directory to be able to configure this setting. Therefore if you try the above Exchange Management Shell cmdlet it will fail with the following error:

Active Directory operation failed on dc-name. This error is not retriable. Additional information: Insufficient access rights to perform the operation.

Active directory response: 00002098: SecErr: DSID-03150BB9, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0

    + CategoryInfo          : NotSpecified: (0:Int32) [Set-AdSiteLink], ADOperationException

    + FullyQualifiedErrorId : ADC691A4,Microsoft.Exchange.Management.SystemConfigurationTasks.SetAdSiteLink

The issue comes down to the fact that the Exchange Trusted Subsystem user account does not have permissions to the delivContLength attribute on the AD site link that you are trying to change. Therefore to make this setting in Exchange you need first to set the correct permissions in AD.

To set the correct permissions open Active Directory Sites and Services (if running Windows 2008 R2 or later) or ADSIEdit if using an earlier version of Windows. Expand Sites and Services to find Sites > Inter-Site Transports and right-click the IP container and choose Properties and change to the Security tab:

image

In ADSIEdit connect to the Configuration well known Naming Context and expand to CN=Configuration… > CN=Sites > CN=Inter-Site Transports and right-click CN=IP. Again select Properties and change to the Security tab:

image

Once in the Security tab click Advanced, click Add and type Exchange Trusted Subsystem. In the Permission Entry for IP dialog that appears once you click OK select the Properties tab and then select Descendant Site Link Objects in the Apply To box:

image

In this dialog find the Write delivContLength permission and click Allow.

Click OK enough times to close all the dialog boxes and windows and you have now granted Exchange the permission to set the MaxMessageSize property on any (and all future) AD site links that you have or may create.

Hosting Exchange 2010 and Issues With Duplicate Contacts

Posted on 1 CommentPosted in 2007, 2010, active directory, adsiedit, domain, exchange, hosting, hybrid, policy

When you are creating a hosted Exchange system using the Exchange 2010 On Premises product (not the /hosting version of the product) it is likely that if two or more of your customers create a mail contact in the global address list (GAL) for the same external email recipient they will see some issues with email addressing.

For example, you are hosting Exchange for northwind.com and fineartschool.net within one Exchange organization. Both these companies have a professional relationship with greg@fabrikam.com and so want to create a contact for him in the GAL. The first of your clients to create the contact will be successful, but any future client receives the following error when they attempt to create the contact:

New-MailContact -Name “Greg (Fabrikam)” -ExternalEmailAddress greg@fabrikam.com -OrganizationalUnit FineArtSchool
The proxy address “SMTP:greg@fabrikam.com” is already being used by “isp.corp/Hosted/Northwind/Greg (Fabrikam)”. Please choose another proxy address.
    + CategoryInfo          : NotSpecified: (…) :ADObjectId) [New-MailContact], ProxyAddressExistsException
    + FullyQualifiedErrorId : B333D21C,Microsoft.Exchange.Management.Recipient
   Tasks.NewMailContact

The work around is to specify a unique proxy address, as the default proxy address (the contacts actual email address) is already being used:

New-MailContact -Name “Greg (Fabrikam)” -ExternalEmailAddress greg@fabrikam.com -OrganizationalUnit FineArtSchool -PrimarySmtpAddress greg@fineartschool.net

Of course Greg’s email address is greg@fabrikam.com (his external email address) and not greg@fineartschool.net (his proxy or primary SMTP address so far as Fine Art School have configured) and if this client sends an email to Greg and they select Greg from the GAL it will go to his external email address but will look like it has gone to his proxy address. That is, Greg will receive the email but if he looks at the address it was sent to it will say greg@fineartschool.net.

Send an email to two people in external organizations, one being greg@fabrikam.com, and hit Reply All and Greg will appear as greg@proxyaddress and not greg@fabrikam.com. Emails in reply will go to Greg via the hosting company and not direct to Greg. This also has the side affect of showing presence (from Microsoft Lync) as being unavailable as the email is using the wrong email address.

The underlying problem is that though the email is being delivered to the external address (targetAddress attribute in Active Directory) it is being stamped with the primary SMTP address (proxyAddresses in Active Directory) in the P2 header. The P2 header is used to generate the Reply address.

So how do you fix this? The obvious way at first glance is to modify active directory and change the proxyAddresses value back to the correct value – but this does not work (as two objects cannot have the same proxy address). Regardless of the fact that the two mail contacts both have the same targetAddress and proxyAddresses, Exchange Transport detects a problem and reports the error “More than one Active Directory object is configured with the recipient address greg@fabrikam.com. Messages to this recipient will be deferred until the configuration is corrected in Active Directory” in the event log on the first Hub Transport server that sees the message.

So without writing your own transport agent, you need to route all outbound email via an Edge Transport server and configure the Address Rewriting agent. You need to create an address rewrite rule for every contact that is created within your hosted organization once the second contact is created. So in your mail contact provisioning application you need to trap the duplicate proxy address error above, reissue the mail contact creation step, this time with a unique primary SMTP address in the hosted clients domain and then at the same time make an address rewrite rule on your Edge Transport server.

New-AddressRewriteEntry -Name “Greg – Fabrikam – HosterFineArtSchool” -InternalAddress greg@fineartschool.net -ExternalAddress greg@fabrikam.com -OutboundOnly $true

Note that rewrite rules are cached for four hours, so unless you restart the MSExchangeTransport service your rewrite rules will not take effect until four hours have gone by.