Speaking at TechEd Europe 2014

Posted on 4 CommentsPosted in certificates, cloud, EOP, exchange, exchange online, Exchange Online Protection, GeoDNS, hybrid, IAmMEC, journaling, mcm, mcsm, MVP, Office 365, smarthost, smtp, starttls, TechEd, TLS, transport

I’m please to announce that Microsoft have asked me to speak on “Everything You Need To Know About SMTP Transport for Office 365” at TechEd Europe 2014 in Barcelona. Its going to be a busy few weeks as I go from there to the MVP Summit in Redmond, WA straight from that event.

image

My session is going to see how you can ensure your migration to Office 365 will be successful with regards to keeping mail flow working and not seeing any non-deliverable messages. We will cover real world scenarios for hybrid and staged migrations so that we can consider the impact of mail flow at all stages of the project. We will look at testing mail flow, SMTP to multiple endpoints, solving firewalling issues, and how email addressing and distribution group delivery is done in Office 365 so that we always know where a user is and what is going to happen when they are migrated.

Compliance and hygiene issues will be covered with regards to potentially journaling from multiple places and the impact of having anti-spam filtering in Office 365 that might not be your mail flow entry point.

We will consider the best practices for changing SMTP endpoints and when is a good time to change over from on-premise first to cloud first delivery, and if you need to maintain on-premises delivery how should you go about that process.

And finally we will cover troubleshooting the process should it go wrong or how to see what is actually happening during your test phase when you are trying out different options to see which works for your company and your requirements.

Full details of the session, once it goes live, are at http://teeu2014.eventpoint.com/topic/details/OFC-B350 (Microsoft ID login needed to see this). Room and time to be announced.

The New Rights Management Service

Posted on 3 CommentsPosted in aadrm, active directory, certificates, cloud, compliance, dirsync, exchange, exchange online, https, hybrid, journal, journaling, mcm, mcsm, microsoft, Office 365, Outlook, pki, policy, rms, smarthost, transport, unified messaging, voicemail

This blog is the start of a series of articles I will write over the next few months on how to ensure that your data is encrypted and secured to only the people you want to access it, and only for the level of rights you want to give them.

The technology that we will look at to do this is Microsoft’s recently released Windows Azure Active Directory Rights Management product, also known as AADRM or Microsoft Rights Management, or “the new RMS”.

In this series of articles we will look at the following:

The items above will get lit up as the article is released – so check back or leave a comment to this post and I will let you know when new content is added to this series.

What is “rights management”

Simply this is the ability to ensure that your content is only used by whom you want it to be used by and only for what you grant. Its known in various guises, and the most common guise is Digital Rights Management (DRM) as applied to the music and films you have been downloading for years.

With the increase in sharing music and other mp3 content in the last ten plus years, the recording companies and music sellers started to protect music. It did not go down well, and I would say this is mainly because the content was bought and so the owner wanted to do with it as they liked – even if what they liked was legal they were limited from doing so. I have music I bought that I cannot use because the music retailer is out of business or I tried to transfer it too many times. I now buy all my music DRM free.

But if the content is something I created and sold, rather than something I bought I see it very differently. When the program was running I was one of the instructors for the Microsoft Certified Master program. I wrote and delivered part of the Exchange Server training. And following the reuse of my and other peoples content outside of the classroom, the content was rights protected – it could be read only by those who I had taught. Those I taught think differently about this, but usually because the management of getting a new copy of the content when it expires!

But this is what rights management is, and this series of articles will look at enabling Azure Active Directory Rights Management, a piece of Office 365 that if you are an E3 or E4 subscriber then you already have, and if you have a lower level of subscription or none at all you can buy for £2/user/month and this will allow you to protect the content that you create, that it can be used by only those you want to read it (regardless of where you or they put it) and if you want it can expire after a given time.

In this series we will look at enabling the service and connecting various technologies to it, from our smartphones to PC’s to servers and then distributing our protected content to whom needs to see it. Those who receive it will be able to use the content for free. You only pay to create protected content. We will also look at protecting content automatically, for example content that is classified in a given way by Windows Server or emails that match certain conditions (for example they contain credit cards or other personally identifiable information (PII) information such as passport or tax IDs) and though I am not a SharePoint guru, we will look at protecting content downloaded from SharePoint document libraries.

Finally we will look at users protecting their own content – either the photographs they take on their phones of information they need to share (documents, aka using the phones camera as a scanner) or taking photos of whiteboards in meetings where the contents on the board should not be shared too widely.

Stick around – its a new technology and its going to have a big impact on the way we share data, regardless of whether we share it with Dropbox or the like or email or whatever comes next.

Secret NSA Listening Ports in Exchange Server 2013? Of Course Not…

Posted on Leave a commentPosted in 2013, Edge, exchange, firewall, IAmMEC, iis, networking, powershell, transport

But what do those extra ports in Exchange Server 2013 that are listening actually do.

If you bring up a command prompt on an Exchange Server 2013 machine and run netstat –ano | find “:25”. You will get back a list of IP addresses that are listening on any port starting 25. The last number on the line is the process ID for that listening port. So for Mailbox only role servers you are interested in the row that shows 0.0.0.0:25 and for a multirole (CAS and Mailbox) server, the row that shows 0.0.0.0:2525 as shown:

image

Above you can see that process 21476 is listening on 2525 and as this is a multirole server this process ID will be EdgeTransport.exe – you can verify this in Task Manager if you want.

Repeat the netstat cmd, this time for the process ID you have selected: netstat –ano | find “xxxxx” where xxxxx is the process ID for EdgeTransport.exe, as shown:

image

You will now see that EdgeTransport.exe is listening in on a range of ports for both IPv4 and the same ports for IPv6. These ports are 25 or 2525, which is used by 2007 or 2010 Edge role servers, or by other 2007/2010 Hub Transport role servers or by other 2013 Mailbox role servers or by 2013 CAS role servers to send emails to this server via SMTP. Port 465 is the port that 2013 CAS servers proxy authenticated SMTP connections to that they receive on port 587 from mail clients. But what about port 29952 in my example (and on your servers a different port) which changes each time the service is restarted?

If you do netstat again just for this port you will see something like the following:

image

This shows that nothing is connected currently to these ports, and so they seem to be doing nothing. But if on a different Exchange 2013 server you do some viewing of the transport queues then these ports will start to show some activity.

On a different server in my environment I ran Get-Queue –Server remoteservername. If I do this on the local server, then nothing special happens as Exchange does not need to connect to these ports, but if it is run from a different server and I ask it to show the queue on the first server that we have been looking at above, then these ports become used:

image

image

Above we can see Exchange Management Shell on mail5 connecting to mail4 (the original server in this blog post). The second picture from mail4 shows that port 29952 have received a connection from the IPv6  address of mail5 and specifically from port 65172 on that remote server.

If I look finally at the second server in this exercise and see what process is connecting from port 65172 (and again, your ports will be different) I see that process 15156 is doing this (the process ID is the last column in the output)

image

Taking this process ID to Task Manager, I see process 15156 is the IIS Worker Process, which is the process that PowerShell connects to to do its work.

image

Therefore, the random and changing port that EdgeTransport.exe listens on is nothing to do with PRISM, but all to do with your management of remote queues.

Ensuring Email Delivery Security with Exchange 2013

Posted on 4 CommentsPosted in 2007, 2010, 2013, encryption, exchange, IAmMEC, TLS, transport

To force Exchange 2013 to guarantee the secure delivery of a message can be done a few different ways. In this version of the product and in previous versions it was possible to create a send connector for a given domain and enable Mutual TLS on the connector. Then all messages to the domain(s) that this connector serviced would need to travel over a TLS connection where the certificate at both ends was completely valid (i.e. valid regards the date, had the correct subject or SAN for the domain, was issued by a trusted certificate authority etc.). In previous versions (2007 to 2010 again) it was possible to enable Domain Secure and add another level of checks to the Mutual TLS session. Domain Secure does not work in Exchange 2013.

And great though these methods of transport security are, they are limited in that they are difficult to set up (require good knowledge of certificates) and needs to be properly configured at both ends of the connection. They are also limited in that they will only secure email to the selected domains. If you need to send a “top secret” email to someone, you don’t really want to have to configure a connector at both ends and force all email for that domain down the same path.

So, in Exchange 2013 you can create a transport rule to force the connection to use TLS, and if TLS fails then have the message queue on the sender until it retries and eventually expires. If TLS is never available, the message never goes out of Exchange – or so it would be if all you read was the description in the documentation!

The RouteMessageOutboundRequireTLS transport rule action (or the Secure the message with > TLS encryption option in the ECP transport rules wizard) allows you to craft a rule for any condition (for example the subject or body contains any of these words: top secret) which will require the email to use an encrypted session for the delivery outside of Exchange Server. Note that for this to work the TLS session does not need to be protected by a given certificate or valid etc., it just needs the receiving SMTP Server to offer STARTTLS and for the encryption to work.

And it needs a source server for sending the message in every Active Directory site within the organization. Currently (as of CU2 for Exchange 2013) if you send a message from a site that does not contain a send connector that can handle the message to the internet then Exchange will pass it to a site that can, but the source transport server will now not enforce the TLS requirement and will send the message unprotected if STARTTLS is not offered.

So if you want to guarantee the use of TLS for certain types of message use the RouteMessageOutboundRequireTLS transport rule condition and ensure that you do not need to do cross site delivery of messages to reach a send connector source server to delivery the message to the internet.

Moving Exchange Online Protection Junk Mail to the Junk Email Folder

Posted on Leave a commentPosted in 2007, 2010, 2013, active directory, cloud, Edge, EOP, exchange, exchange online, FOPE, mcm, mcsm, spam, transport

If you use Exchange Online Protection (EOP) to filter your email in the cloud and to remove spam and malware before onward delivery to you, and if you use Exchange 2007 or later on-premises, then you need to configure Exchange to move detected spam to the Junk Email folder in Outlook.

By default EOP detects two levels of spam (malware is automatically removed) and tags them. In Exchange, you need to use Transport Rules to move these emails to the users Junk Email folder. If you wish for EOP throw away junk emails of either detection level (i.e. high junk is discarded) then you still need to configure these transport rules to move the remaining detected junk email into the Junk Email folder in Outlook. If you place all EOP detected spam in the quarantine or throw it away, you should still create these transport rules as it means they are in place for any future changes you make at EOP.

If you use Exchange Online (part of Office 365) then you do not need to create these rules as they already exist (though you cannot see them in any admin tools you have).

The Transport Rules To Create

The following four transport rules need to be created on your Exchange organization. These four rules get created with the highest priority, moving all existing rules below them. They set the Spam Confidence Level of the message to 6 in these examples, though this should be set to a value that exceeds the SCLJunkThreshold organization wide setting for your Exchange Organization, as any email that exceeds this value is placed into the Junk Email folder upon delivery to the users mailbox.

New-TransportRule "Move EOP Detected Spam (SFV:SPM) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SFV:SPM" -SetSCL 6 -Priority 0

New-TransportRule "Move EOP Detected Spam (SFV:SKS) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SFV:SKS" -SetSCL 6 -Priority 1

New-TransportRule "Move Personal Block List Spam (SFV:SKB) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SFV:SKB" -SetSCL 6 -Priority 2

New-TransportRule "Move Bulk Email (SFV:SKB) to Junk Email Folder" -HeaderContainsMessageHeader "X-Forefront-Antispam-Report" -HeaderContainsWords "SRV:BULK" -SetSCL 6 -Priority 3

Once these four rules are run, AD replication will need to be allowed to happen and then all Exchange Servers will action this rule for any email that EOP thinks is spam. The default value for SCLJunkThreshold is 4, so as long as the rules set the SCL value to greater than this value it should work. Use Get-OrganizationConfig | FL SCLJunkThreshold to get this value for your organization.

What the EOP Message Tags Mean

The X-Forefront-Antispam-Report header in the email contains many values. See http://technet.microsoft.com/en-us/library/dn205071(v=exchg.150).aspx for what EOP thinks about the spam rating of the message.

  • SFV – means Spam Filtering Verdict
  • SFV:SFE – Originated from a Safe Sender (EOP learns Outlook safe senders due to Windows Azure Directory Sync)
  • SFV:BLK – Originated from a Blocked Sender
  • SFV:SPM – Spam
  • SFV:SKB – Emails where the sender is on the users personal block list as configured in OWA or Outlook Junk Email settings
  • SFV:SKS – (SKIP) The message was marked as spam prior to being processed by the spam filter. This includes messages where the message matched a Transport rule to automatically mark it as spam and bypass all additional filtering
  • SFV:NSPM – Not Spam

Article updated in June 2016 to include new headers and go from two rules to four

Removing Edge Subscription When Exchange 2013 Installed

Posted on 6 CommentsPosted in 2007, 2010, 2013, Edge, exchange, federation, IAmMEC, mcm, mcsm, smarthost, transport

Exchange 2013 does not have an Edge role (at the time of writing – Aug 2013). It is possible to use Exchange 2010 SP3 and install the Edge role should you need one.

There is a problem though when it comes to removing the Edge Subscription between an organization that contains Exchange 2013 servers and the Exchange 2010 Edge Server. To remove the subscription on the Edge server role you run Remove-EdgeSubscription servername and this removes both the subscription and any subscribed objects from the local AD LDS database on that Edge Server. But if any of these subscribed objects where created on Exchange 2013 after it was installed, then they will have an ExchangeVersion equal to 0.20 (15.0.0.0). The Exchange 2010 SP3 Remove-EdgeSubscription cannot process this object and so fails with:

Remove-EdgeSubscription : Can’t make this change because the object’s ExchangeVersion property is 0.20 (15.0.0.0), which is not supported by the current version 0.1 (8.0.535.0). You will need a newer version of Exchange to make this change. Property Name: ExchangeVersion
At line:1 char:24
+ Remove-EdgeSubscription <<<<  edge2 -Verbose
+ CategoryInfo          : NotSpecified: (0:Int32) [Remove-EdgeSubscription], DataValidationException
+ FullyQualifiedErrorId : D8A49A14,Microsoft.Exchange.Management.SystemConfigurationTasks.RemoveEdgeSubscription

The way to fix this is to find and manually remove the object with an ExchangeVersion of 0.20 (15.0.0.0) from the AD LDS database and then repeat the Remove-EdgeSubscription cmdlet – as that should now work (unless you have two or more objects with the higher version number to locate and delete).

  1. To find objects with an ExchangeVersion greater than “0.1 (8.0.535.0)”, which is the version Exchange 2010 will process, open ADSIEdit on the Edge server.
  2. Right-click the ADSI Edit node at the top of the window and choose Connect to…
    image
  3. In the Connection Settings dialog (shown above), change the Select a well known Naming Context to Configuration and type the local server name and the AD LDS port in the Select or type a domain or server field. The server:port value should be EDGESERVERNAME:50389
    image
  4. Expand the tree view until you reach CN=First Organization,CN=Microsoft Exchange,CN=Services,CN=Configuration,CN={AD258B4D-CCB4-4125-80C1-7B73CE066341}
    image
  5. You now need to look at each object, starting with those you remember having created since 2013 was installed, for an object who’s msExchVersion value is not 4535486012416. For example, in the below screenshot I have an accepted domain made due to Federation in Exchange 2013. This object (under CN=Accepted Domains,CN=Transport Settings,…) has a value of 88218628259840.
    image
  6. To validate that this is the correct object to manually delete, from the Exchange Management Shell on the Edge server, enter Get-Object | FT name,ExchangeVersion where Object is the cmdlet that you are looking to query – in my case it would be Get-AcceptedDomain.
    image
  7. As you can see, this object has a newer ExchangeVersion and so it is (at least) this object that is stopping Exchange EdgeSync from being removed.
  8. Manually delete this object in ADSI Edit (it is safe to do this as it will resync from the Exchange organization if you recreate Edge Subscription later). Do not delete it from the Active Directory with ADSI Edit – just from AD LDS. Take care to only delete this object and not the parent object.
  9. Once this object is gone, try Remove-EdgeSubscription servername again. If this is the only object, then the Edge Subscription will be removed in the Edge Server

You can now carry on with whatever it was that you were doing that required a removal of the Edge Subscription.

Journal Alternative Mailbox and No Inbox Rules

Posted on 1 CommentPosted in 2013, compliance, exchange, exchange online, journal, journaling, mcm, mcsm, ndr, Office 365, rules, transport, transport agent

In the event of your journal mailbox going offline, any journal reports destined for these mailboxes will queue. After two days (though this time is the expiry time for messages in your Exchange organization, so may be different) the message will expire and an NDR sent to the sender of the journal report. The problem is that the journal report was not sent by anyone – the From address is <>. So no NDR is generated and the journal report is lost.

There is the JournalReportNdrTo property of TransportConfig that allows you to set who will receive these NDR’s.

Set-TransportConfig -JournalingReportNdrTo journalndr@mcmemail.co.uk

Once this value is set this mailbox should be monitored occasionally and any NDR’s opened and the containing message (the journal report) resent so that it goes back to the (now working) journal mailbox.

In Exchange 2013 this NDR mailbox is never the subject of journaling nor do any inbox rules run against this mailbox – even if this mailbox is mentioned in a journal rule of if the mailbox has inbox rules associated with it. When you set this value in your Exchange 2013 organization you get the following warning:

WARNING: Any mail to JournalingReportNdrTo mailbox will not be journaled and it will not honor transport and mailbox rules settings. It is recommended to create a dedicated mailbox for JournalingReportNdrTo setting or set it to an external address.

Or if you set it in Exchange Control Panel then the following popup appears:

image

The warning also mentions that Transport Rules do not fire for this mailbox, but that is not what I have seen – though it might be that specific transport rules do not get actioned, but others do. Inbox rules and Journal Rules are not processed.

Therefore it is very important that you do not use a standard mailbox as the target for JournalReportNdrTo, as this mailbox will have all its outbound emails missing from any journal it should be stored it (and this would be a compliance issue) and the user will get bothered that their email rules in Outlook are not working.

The problem is that this is not the case in Exchange 2010, so if you have set the JournalReportNdrTo property in the past on a mailbox, and then migrated that mailbox to 2013 you will not be warned, but you will find that upon migration to 2013 your inbox rules stop working and if you look in the journal mailbox you will not find messages send from this mailbox. Therefore create a mailbox specifically for journal NDR’s before you migrate to Exchange 2013.

Domain Secure and Edge Servers

Posted on Leave a commentPosted in 2007, 2010, 2013, certificates, cloud, exchange, firewall, smarthost, smtp, transport

I was asked a question recently on the Microsoft Certified Master course for Exchange 2010 and was told that the answer was not clearly written up on the internet. So I thought I would write this blog post. The question was based on the idea that Domain Secure worked from a Hub Transport server in the classroom lab but not when mail flow went via an Edge server.

Domain Secure is end to end security, it cannot have anything in the middle – i.e. it cannot go via an Exchange Edge server, an Exchange 2013 Frontend Server or a third party SMTP relay.

The SMTP client in the connection (the send connector host) needs to connect to the SMTP server (the receive connector host) and swap certificates and prove the other side is who the other side say they are – i.e. mutual authentication. Also the domains must match the TLS list in TransportConfig (TLSSendDomainSecureList and TLSReceiveDomainSecureList). Therefore anything in the middle will offer a different certificate and so Domain Secure fails.

If there is a middle party and you want to do mutual authentication (i.e. swap certs to prove who you are), with one party offering their cert and not the cert of the final recipient domain (i.e. mail.messaging.microsoft.com or postini.com etc.) then use TLSAuthLevel and the DomainValidation option on the send connector (an SP1 addition to Exchange 2010). No green ticky ticky though.

Edge can do Domain Secure though. But Edge needs to be the starting point, i.e. the host of the send connector. So configure Domain Secure on the Edge (i.e. set the certificates and correct firewall settings) and ensure that the send connector for Domain Secure has the Edge server as the source. Ensure Edge Domain Secure receive connector is the target for inbound as well if you want it to work both ways. And of course you need working EdgeSync so hubs can deliver to Edge so that Edge can deliver emails for you.

Enabling Exchange 2013 to Filter OneNote and Publisher Files

Posted on 1 CommentPosted in 2013, 64 bit, exchange, IFilter, owa, transport

Exchange Server 2013 includes the Search Foundation product to index and search most of the file types that needed IFilters installed for in previous versions including PDF files, so the Adobe IFilter is no longer needed. That said, it does not filter OneNote and Microsoft Publisher files.

To filter these files so that you can search them as part of your mailbox search, include them in discovery and compliance searches and to write transport rules that can act on the contents of these files you need to install the Microsoft Office Filter Pack 2010 and Service Pack 1 for Microsoft Office Filter Pack 2010 (KB 2460041) 64-bit Edition and then, once installed, set a few registry keys. The following script does the registry key steps for you, and needs running on all your Exchange 2013 Mailbox Servers. See http://marksmith.netrends.com/Lists/Posts/Post.aspx?ID=93 for steps to install the PDF filter on Exchange 2010 if you need to do that. This script is based on the work in that post and the original scripts from Microsoft to configure the IFilters in Exchange 2010 RTM.

Script for Install-IFilters.ps1

Download the Install-IFilters.ps1 here (which is a zip containing some test files as well – see below for testing steps. The PowerShell script is also shown below:

   1: # Script to enable indexing of OneNote and Microsoft Publisher filtering in Exchange 2013

   2: # Written by Brian Reid, C7 Solutions Ltd. www.c7solutions.com 14:27 21/11/2012

   3: # Based on a script by Mark Smith from Capex Global that installed PDF filtering for Exch2010

   4: # Note PDF filtering is included in Exchange 2013 and not needed as an extra install

   5:

   6: # The Microsoft Office 2010 Filter Pack needs installing before running this script: http://www.microsoft.com/en-us/download/details.aspx?id=17062

   7: # The Microsoft Office 2010 Filter Pack SP1 (x64) needs installing before running this script: http://www.microsoft.com/en-us/download/details.aspx?id=26604

   8: # This script has not been tested with the Microsoft Office Filter Pack 2013 release as it was not available at time of writing

   9:

  10: # This script will restart the Transport service and Microsoft Filtering Management Service. Mail-flow might be affected

  11: # by the first of these restarts. Existing items in the store will not be indexed

  12:

  13: $iFilterDirName = "C:\Program Files\Common Files\Microsoft Shared\Filters\"

  14:

  15: $KeyParent = "HKLM:\SOFTWARE\Microsoft\ExchangeServer\v15\HubTransportRole"

  16: $CLSIDKey = "HKLM:\SOFTWARE\Microsoft\ExchangeServer\v15\HubTransportRole\CLSID"

  17: $FiltersKey = "HKLM:\SOFTWARE\Microsoft\ExchangeServer\v15\HubTransportRole\filters"

  18:

  19: # Filter DLL Locations

  20: $ONEFilterLocation = $iFilterDirName + "\ONIFilter.dll"

  21: $PUBFilterLocation = $iFilterDirName + "\PUBFILT.dll"

  22:

  23: # Filter GUIDs

  24: $ONEGuid    ="{B8D12492-CE0F-40AD-83EA-099A03D493F1}"

  25: $PUBGuid    ="{A7FD8AC9-7ABF-46FC-B70B-6A5E5EC9859A}"

  26:

  27:

  28: # Create CLSID and filters root registry keys if they do not exist

  29: Write-Host -foregroundcolor Green "Creating parent registry keys"

  30:

  31: New-Item -Path $KeyParent -Name CLSID -ErrorAction SilentlyContinue | Out-Null

  32: New-Item -Path $KeyParent -Name filters -ErrorAction SilentlyContinue | Out-Null

  33:

  34:

  35: # Create CLSIDs

  36: Write-Host -foregroundcolor Green "Creating CLSIDs..."

  37:

  38: New-Item -Path $CLSIDKey -Name $ONEGuid -Value $ONEFilterLocation -Type String | Out-Null

  39: New-Item -Path $CLSIDKey -Name $PUBGuid -Value $PUBFilterLocation -Type String | Out-Null

  40:

  41: # Set Threading model

  42: Write-Host -foregroundcolor Green "Setting threading model..."

  43:

  44: New-ItemProperty -Path "$CLSIDKey\$ONEGuid" -Name "ThreadingModel" -Value "Both" -Type String | Out-Null

  45: New-ItemProperty -Path "$CLSIDKey\$PUBGuid" -Name "ThreadingModel" -Value "Both" -Type String | Out-Null

  46:

  47: # Set Flags

  48: Write-Host -foregroundcolor Green "Setting Flags..."

  49: New-ItemProperty -Path "$CLSIDKey\$ONEGuid" -Name "Flags" -Value "1" -Type Dword | Out-Null

  50: New-ItemProperty -Path "$CLSIDKey\$PUBGuid" -Name "Flags" -Value "1" -Type Dword | Out-Null

  51:

  52: # Create Filter Entries

  53: Write-Host -foregroundcolor Green "Creating Filter Entries..."

  54:

  55: New-Item -Path $FiltersKey -Name ".one" -Value $ONEGuid -Type String | Out-Null

  56: New-Item -Path $FiltersKey -Name ".pub" -Value $PUBGuid -Type String | Out-Null

  57:

  58: # Setting permissions

  59: Write-Host -foregroundcolor Green "Granting NETWORK SERVICE read access to $KeyParent and child keys "

  60: $acl = Get-Acl $KeyParent

  61: $rule = New-Object System.Security.AccessControl.RegistryAccessRule ("NETWORK SERVICE","ReadKey","Allow")

  62: $acl.SetAccessRule($rule)

  63: $acl | Set-Acl -Path $KeyParent

  64:

  65: # Restarting required services

  66: Write-Host -foregroundcolor Green "Stopping Microsoft Exchange Transport service (this takes a few minutes)"

  67: Stop-Service "Microsoft Exchange Transport" | Out-Null

  68: Write-Host -foregroundcolor Green "Stopping Microsoft Filtering Management Service"

  69: Stop-Service "Microsoft Filtering Management Service" | Out-Null

  70: Write-Host -foregroundcolor Green "Starting Microsoft Exchange Transport service (this takes a few minutes)"

  71: Start-Service "Microsoft Exchange Transport" | Out-Null

  72: Write-Host -foregroundcolor Green "Starting Microsoft Filtering Management Service"

  73: Start-Service "Microsoft Filtering Management Service" | Out-Null

Testing Attachment Filtering

The download above contains the script and some test files for different document types. To test just create a transport rule where:

    • The sender is your mailbox.
  • Any attachment’s content includes “Testing IFilters”. (the files in the download include these words)

 

  • Generate an incident report and send it to your mailbox. Incident Reports are advanced transport rule actions.

 

If you create this rule before you run the script then you will get incident reports for the TXT file, DOCX file and the PDF file (note, you did not need to install the Adobe IFilter to get this functionality). But sending the ONE file and the PUB file before running the above script, even though you have the Microsoft Filter Pack installed, will not generate an incident report.

Once you have run the script, email yourself the ONE and PUB files and both should generate an incident report. From now on trasnport rules will process OneNote and Microsoft Publisher documents correctly, including any that match any Data Loss Prevention rules that you have enabled.

Missing the Message Tracking Log Explorer in Exchange 2013? Not anymore…

Posted on 8 CommentsPosted in 2013, exchange, exchange online, management, powershell, proxy, transport

Exchange 2013 has removed a number of user interfaces that existed in Exchange 2010, one of them being the Message Tracking toolbox utility:

image

In Exchange 2013 you can search for an individual messages in the Exchange Control Panel (https://servername/ecp) by selecting mail flow > delivery reports. But this tool requires you to specify the source mailbox and limits the answers somewhat, especially if your administrators mailbox is still on Exchange 2010 and you used https://servername/ecp/?exchclientver=15 to access Exchange 2013 ECP:

image

If you want to search across lots of mailboxes then your options are limited to Exchange Management Shell and using PowerShell. And with the considerably increase in internal monitoring emails that get sent around inside Exchange 2013, the logs are busy with all the Delivery Probe emails:

image
Get-MessageTrackingLog -Start “Nov 2 2012”

So to the rescue comes a PowerShell 2.0 feature called Out-GridView. To run a message tracking log in a user interface start in Exchange Management Shell with a simple Get-MessageTrackingLog cmdlet and output the results to Out-GridView

image
Get-MessageTrackingLog -ResultSize Unlimited -Start “Nov 2 2012” | Out-GridView

Now that you have the results in a grid you can add filtering, for example the below shows the tracking logs on my server once I filter out all the “probe” emails that are used by the Managed Availability feature to ensure that the server is operating correctly:

image

So although you need a bit of a command to start with, Get-MessageTrackingLog -ResultSize Unlimited -Start “Nov 2 2012” | Out-GridView, and need to include –ResultSize Unlimited to ensure you have all the results before you start filtering, this is a way to reduce some of the time to do tracking logs as the filtering and scrolling/resizing and ordering steps are managed for you which you do not get in the shell.

And of course, there are loads of things you can use Out-GridView for in Exchange 2013. Some of these include Get-Mailbox, Get-MailboxDatabase, Get-whatever you like that returns multiple rows of data.

Create Shadow Redundancy Cross Forest in Exchange 2010

Posted on Leave a commentPosted in 2010, IAmMEC, mcm, transport

 

Send connector cross forest shadow redundancy

New-SendConnector ToTailspin -AddressSpaces SMTP:tailspin.com -SmartHosts mail.tailspin.com -ProtocolLoggingLevel verbose -DNSRoutingEnabled $False -SmartHostAuthMechanism ExternalAuthoritative
Get-SendConnector ToTailspin | Add-ADPermission -user "MS Exchange\Externally Secured Servers" -ExtendedRights ms-Exch-SMTP-Send-XShadow

Receive connector cross forest shadow redundancy

New-ReceiveConnector FromFabrikam -RemoteIPRanges 192.168.100.1 -Bindings 0.0.0.0:25 -ProtocolLoggingLevel verbose -Banner "220 Tailspin XShadow SMTP Server" -AuthMechanism ExternalAuthoritative 
Get-ReceiveConnector FromFabrikam | Add-ADPermission -user "MS Exchange\Externally Secured Servers" -ExtendedRights ms-Exch-SMTP-Accept-XShadow

Creating GeoDNS with Amazon Route 53 DNS

Posted on 3 CommentsPosted in 2013, cloud, exchange, GeoDNS, https, load balancer, mcm, microsoft, MX, networking, owa, smtp, transport

UPDATE: 13 Aug 2014 – Amazon Route 53 now does native GeoDNS within the product – see Amazon Route 53 GeoDNS Routing Policy

A new feature to Exchange 2013 is supported use of a single namespace for your global email infrastructure. For example mail.contoso.com rather than different ones for each region such as uk-mail.contoso.com; usa-mail.contoso.com and apac-mail.contoso.com.
GeoDNS means that you are given the IP address of a server that is in or close to the region that you are in. For example if you work in London and your mailbox is also in London then most of the time you will want to be connected to the London CAS servers as that gives you the best network response. So in Exchange 2010 you would use your local URL of uk-mail.contoso.com and if you used the others you would be told to use uk-mail.contoso.com. For GeoDNS support you use mail.contoso.com and as you are in the UK you get the IP address of the CAS array in London. When you travel to the US (occasionally) you would get the US CAS array IP address, but this CAS array is able to proxy your OWA, RPC/HTTP etc traffic to the UK mailbox servers.
The same is true for email delivery via SMTP. Email that comes from UK sourced IP addresses is on balance a statistical likelihood that it is going to the UK mailbox. So when you look up the MX record for contoso.com from a UK company you get the UK CAS array and the email gets delivered to the CAS array that is in the same site as the target mailbox. If the email is for a user in a different region and it hits the UK CAS array then it is proxied to the other region seamlessly.
GeoDNS is a feature provided by some high-scale DNS providers, but not something Amazon Web Services (AWS) Route 53 provides – so how do I configure GeoDNS with Amazon Web Services (AWS) Route 53 DNS Service?
Quite easily is the answer. Route 53 does not offer GeoDNS but does offer DNS that directs you towards the closest AWS datacentre. If your datacentres are in regions similar to AWS then the DNS redirection that AWS offers is probably accurate.
To set it up, open your Route 53 DNS console, or move your DNS to AWS (it costs $0.50/month for a zone at time of writing, AWS Route 53 pricing here) and then create your global Exchange 2013 namespace record in DNS:

  1. Click Create Record Set and enter the name. In the below example I’m using geo.c7solutions.com as I don’t actually have a globally distributed email infrastructure!
  2. Select A – IPv4 or if you are doing IPv6 select AAAA.
  3. Set Alias to No and enter the IP address of one of your datacentres
  4. Select the AWS region that is closest to this Exchange server(s) and enter a unique description for the Set ID value.
  5. The entry will look something like this:
    image
  6. Save the Record Set and create additional entries for other regions. For the purposes of this blog I have created geo.c7solutions.com in four regions with the following IP addresses:
    Region IP Address Region
    us-east-1 1.2.3.4 Northern Virginia
    us-west-1 6.7.8.9 Northern California
    eu-west-1 2.3.4.5 Ireland
    ap-northwest-1 3.4.5.6 Singapore
    sa-east-1 4.5.6.7 Sao Paulo
    ap-southeast-1 5.6.7.8 Sydney
  7. The configuration in AWS for the remaining entries looks like the following:
    imageimageimage
  8. And also, once created, it appears like this:
    image

In addition to this blog, I’ve left the record described above on my c7solutions.com DNS zone. So depending upon your location in the world, if you open a command prompt and ping geo.c7solutions.com you should get back the IP address for the AWS region closest to you, and so get back an IP that represents a Exchange resource in your global region. Of course the IP’s I have used are not mine to use and probably will not respond to ping requests – but all you need to do is see it DNS returns the IP above that best matches the region that you are in.
I wrote this blog when in a hotel in Orlando and as you can see from the image below, it returns 1.2.3.4 which is the IP address associated with us-east-1.
image
But when I connected to a server in the UK and did the same ping geo.c7solutions.com I got the following, which show GeoDNS working when equating GeoDNS to AWS Latency DNS.
image
What do you get for your regions? Add comments and let us where you are (approximately) and what region you got. If enough people respond from enough places we can see if AWS can go GeoDNS without massive cost.
[Updated 13 Nov 2012] Added Sydney (ap-southeast-1) and fake IP address of 5.6.7.8
[Updated 27 April 2013] Added Northern California (us-west-1) and fake IP of 6.7.8.9

How To Speed Up Exchange Server Transport Logging

Posted on 1 CommentPosted in 2007, 2010, 2013, exchange, mcm, microsoft, transport

In Exchange 2010 SP1 and later any writing to the transport log files for activity logging (not the transaction logging on the mail.que database) is cached in RAM and written to disk every five minutes.

In a lab environment you might be impacted by this as you might have sent an email and want to check the logs for the information they contain for diagnostic reasons. The problem is you might need to wait five minutes to get this information.

Exchange 2010

To reduce the memory cache time to 30 seconds set the following two entries in the Edge.Transport.exe.config file (found in \Program Files\Microsoft\Exchange Server\v14\bin) within the AppSettings area:

<add key="SmtpSendLogFlushInterval" value="0:00:30" />
<add key="SmtpRecvLogFlushInterval" value="0:00:30" />

The two values above control different log files. Each transport log file has a different setting – so its possible to set Receive Connector protocol logging to a different value from Send Connector protocol logging if you wanted to. Once you make your changes to Edge.Transport.exe.config you need to restart the Microsoft Exchange Transport service for the changes to be picked up.

Here is a list of the properties that I know about that can be changed:

  • SmtpSendLogFlushInterval – Timespan value on how often to write the Send Connector protocol logging log to disk
  • SmtpRecvLogFlushInterval – Timespan value on how often to write the Receive Connector protocol logging log to disk
  • ConnectivityLogFlushInterval – Timespan value on how often the Connectivity log is written to disk.

In addition to the above, which are all timespan values for how often to write to disk, if the memory buffer that contains the log entries fills up then it will be written to disk as well. The default memory buffers are 1MB. So on a very busy server you might find that the log writing is not every five minutes exactly but of a more “random” nature as the buffer is filled. The following settings control the size of the buffer for the above timespans:

  • SmtpSendLogBufferSize
  • SmtpRecvLogBufferSize
  • ConnectivityLogBufferSize

Exchange 2013 CU1 and Later

The process for this version of Exchange is similar, just different log files because of the different services in use.

In Exchange 2013 you have transport services for mailbox (submission and delivery), transport core and CAS (frontend transport). The config files for these services are:

  • EdgeTransport.exe.config (transport core)
  • MSExchangeFrontEndTransport.exe.config (Frontend Transport on the CAS role)
  • MSExchangeDelivery.exe.config (for mailbox delivery on the Mailbox role)
  • MSExchangeSubmission.exe.config (for mailbox submission on the Mailbox role)

You will find these config files in \Program Files\Microsoft\Exchange Server\v15\bin.

 

Shadow Redundancy and Server Outages

Posted on Leave a commentPosted in 2010, exchange, transport

Exchange Server 2010 has a feature that tries to ensure that emails in transport cannot be lost. This feature is called Shadow Redundancy and lots of information on how it works can be found on the Internet.

But what happens if a mailbox server or site is unavailable? Items will queue in a single location, and now this location is a single point of failure. So whilst you have an outage (planned or otherwise), you increase the risk of loss of mail due to a second outage in transport that causes mail.que database corruption.

Let us examine the details by considering one type of outage – other types of problems can occur and generate the same potential results. If I dismount a database in an Active Directory site and then send an email to an Exchange 2010 mailbox on that site the email will queue on an Exchange 2010 Hub Transport server in that site. The queue will be visible with Get-Queue and the queue will go into Retry state. Here is a picture showing the Exchange Management Shell output for one such site:

The first cmdlet shows one email queuing for a mailbox database (the one that is offline), the DeliveryType is MapiDelivery and the NextHopDomain (the next target) is the offline database.

image

The second cmdlet in the above picture shows the effect of a second email being sent. The items in the queue are at 2, and both of these are on FAB-RED-HUB1. Should FAB-RED-HUB1 fail at this point and the mail.que database become corrupt due to this failure, these emails would be lost.

What you cannot see from the screenshot is the effect of Delayed Acknowledgement. Delayed Acknowledgement is the process whereby if a Hub Transport server receives an email from an SMTP server that does not support Shadow Redundancy then it will delay acknowledgement to the message long enough to ensure the message exists on two servers – that is, it has a shadow for the message. In the above example this is not possible as the inbound email is from the internet and is directly into this Active Directory site, so there is nowhere else to send the email. Delayed Acknowledgement is set to 30 seconds by default and so on the arrival of the first message the sending server has their acknowledgement delayed by the full 30 seconds. On the arrival of the second message, as the delivery queue is in retry the Delayed Acknowledgement is not implemented as DelayedAckSkippingEnabled is set to True by default (so if it would take over 30 seconds to deliver or the target queue is in retry then don’t implement a delay as it is likely to be present even after 30 seconds. The problem here is that protection of the first message was 30 seconds, and if the mailbox database (or other failure) was resolved in 30 seconds then you would have delayed the acknowledgement and so protected the message by having not told the previous hop that it was queued. The second message (and all subsequent messages) are immediately added to the outbound queue and are a single point of failure.

Service Pack 1 for Exchange 2010 adds Shadow Redundancy Promotion. This will ensure that the message lives on two transport servers within a site if the NextHopDomain is unavailable. But this is disabled by default.

To enable Shadow Redundancy Promotion, edit the EdgeTransport.exe.config file on all hub transport servers to read True for the ShadowRedundancyPromotionEnabled setting. Once the EdgeTransport.exe.config file is saved then restart the Microsoft Exchange Transport service on all servers. EdgeTransport.exe.config is found in \Program Files\Microsoft\Exchange Server\V14\bin.

This second screenshot shows the effect of enabling Shadow Redundancy Promotion on all my hub transport servers and restarting the transport service on each machine. The screenshot follows on immediately from the above example.

image

In the above you can now see that the queue that did contain the message to the mailbox database (FAB-RED-HUB1\216) is now empty and that their is a shadow queue containing the two messages on FAB-RED-HUB1 instead. FAB-RED-HUB2 (also in the same site) now hold the queue to the offline database. In the event of a transport server failure whilst the database is offline, there will not be a loss of email as the email can be redelivered from the other transport server.