Allowing “Phish” Training Emails in Exchange Online


With the introduction of “Secure by Default” in Exchange Online, where you used to add IPs and other settings such as allow-lists to allow your phish training emails to get through to your users this will no longer work, and shortly it will no longer work to bypass the Exchange Online filters using mail flow (transport) rules.

So to ensure that any third-party phish simulation companies “messages” reach your users, you need to turn on the Advanced Delivery policy. The steps for this are covered below. The IPs and other values you need are obtained from your phish training provider, but this blog post uses (some of) the values from KnowBe4.

The Microsoft “Attack Simulation Training” feature does not require this setup, as this training provider places the messages directly in the mailbox and does not deliver them via SMTP (where they could be filtered out).

The Advanced Delivery policy is found in the new Security Center (https://security.microsoft.com) > Policies & Rules > Threat Policies > Advanced Delivery. Then click the Phishing Simulation tab.

Click Add to add a new policy (as shown) or Edit if you are updating an existing policy. There is only a single policy, so if you use two or more phish training providers you would add all the values to a single policy.

Security Portal showing the Phishing Simulation tab of the Advanced Delivery Policy
The Phishing Simulation tab of the Advanced Delivery Policy

To create the policy you need to add one or more Sending Domains. For KnowBe4 this is psm.knowbe4.com as shown:

Security Portal showing adding a Sending Domain to the Advanced Delivery policy
Adding a Sending Domain to the Advanced Delivery policy

The Sending Domains value can be the P1 domain of the phish simulation emails and from October 2021 it can be the DKIM signing domain as well. You are allowed to add 10 domains to this list, so in cases where the phish simulation provider uses lots of domains it may be that they are all signed with one or two DKIM domains.

The Sending IPs that you need are also provided by your simulation provider. In the case of KnowBe4 they list a US and EU range, though the EU range completely encompasses the US range. You would add all the IPs that your providers send from here as shown:

Security Portal showing adding Sending IPs to the Advanced Delivery Policy
Adding Sending IPs to the Advanced Delivery Policy

Once all the IPs are added you can click Add or Save (if editing) the policy as the Simulation URLs are option (see below).

Security Portal showing adding a new Advanced Delivery Phishing Simulation Policy
Adding a new Advanced Delivery Phishing Simulation Policy

This will result in a view in the admin portal that looks like this:

Security Portal showing the Phishing Simulation Advanced Delivery Policy once edited
The Phishing Simulation Advanced Delivery Policy once edited

Once you have created your policy you can see that you have an edit button. Changes now will result in the flyout prompt showing you want you have changed as shown rather than the whole policy (as shown):

Security portal showing a change to the policy
Security portal showing a change to the policy

And the change above that I have made is to add the optional Simulation URLs. These (for KnowBe4) are provided from your Phish Simulation Admin Portal at KnowBe4, so these values here are just for demo purposes. For example, lets say that your phish simulation provider sends links from email.pretender-domain.net and intranet.pretender-domain.net. You could either add both of these domains, or *.pretender-domain.net or ~pretender-domain.net (the last of these is the root domain and all subdomains). The last of these is preferred as you can only add 10 domains to the Simulation URLs dialog.

Finally once you have added your policy you can go and clean up years of arbitrary settings to allow this in the past. These settings will be found in the following:

Of course, all the above are settings in Security and Compliance portals, and so can be set and modified by PowerShell. Here are the cmdlets to do all the above in one set of cmdlets (you need to connect to Exchange Online for New-TenantAllowBlockListItems and the Security and Compliance PowerShell (IPPSSession) for the PhishSimOverride cmdlets.

Connect-ExchangeOnline -UserPrincipalName admin@domain.com
Connect-IPPSSession -UserPrincipalName admin@domain.com
New-PhishSimOverridePolicy -Name PhishSimOverridePolicy
New-PhishSimOverrideRule -Name PhishSimOverrideRule -Policy PhishSimOverridePolicy -SenderDomainIs psm.knowbe4.com -SenderIpRanges 147.160.167.0/26,52.49.201.246,52.49.235.189,23.21.109.197,23.21.109.212
New-TenantAllowBlockListItems -ListType URL -ListSubType AdvancedDelivery -Entries "~pretender-domain.net" -Allow

And if you are testing, the following will clean up the policy and rule that PowerShell or the portal creates:

Connect-ExchangeOnline -UserPrincipalName admin@domain.com
Connect-IPPSSession -UserPrincipalName admin@domain.com
Remove-PhishSimOverridePolicy -Identity PhishSimOverridePolicy
Remove-TenantAllowBlockListItems -ListType URL -ListSubType AdvancedDelivery -Entries "~pretender-domain.net"

Comments

One response to “Allowing “Phish” Training Emails in Exchange Online”

  1. GW avatar
    GW

    Testing this now. Thank you very much! (Hopefully the URLs are not required for this to work)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.