Configuring Sync and Writeback Permissions in Active Directory for Azure Active Directory Sync

Posted on 47 CommentsPosted in 2008, 2008 R2, 2012, 2012 R2, active directory, ADFS 3.0, Azure, Azure Active Directory, cloud, exchange, exchange online, groups, hybrid, IAmMEC, Office 365, WAP, Web Application Proxy, windows

[Last updated 11th November 2019 – added support for Exchange Server automapping support, which was announced during Microsoft Ignite 2019 and will be supported in the first half of calendar year 2020. This is supported by adding writeback for msExchDelegateListBL and msExchDelegateListLink attributes. The below Exchange Hybrid writeback script has been updated to support these two attributes.]

[This blog post was last updated 5th October 2017 – added support to Exchange Hybrid for msExchDelegateLinkList attribute which was announced at Microsoft Ignite 2017 for the support of keeping auto-mapping working cross on-premises and the cloud]

[Updated 18th June 2017 in advance of the release of AADConnect version 1.1.553.0. This post contains updates to the below scripts to include the latest attributes synced back to on-premises including publicDelegates, which is used for supporting bi-directional sync for “Send on Behalf” of permissions in Exchange Online/Exchange Server hybrid writeback scenarios]

[Update March 2017 – added another blog post on using the below to fix permission-issue errors on admin and other protected accounts at http://c7solutions.com/2017/03/administrators-aadconnect-and-adminsdholder-issues]

Azure Active Directory has been long the read-only cousin of Active Directory for those Office 365 and Azure users who sync their directory from Active Directory to Azure Active Directory apart from eight attributes for Exchange Server hybrid mode. Not any more. Azure Active Directory writeback is now available. This enables objects to be mastered or changed in Azure Active Directory and written back to on-premises Active Directory.

This writeback includes:

  • Devices that can be enrolled with Office 365 MDM or Intune, which will allow login to AD FS controlled resources based on user and the device they are on
  • “Modern Groups” in Office 365 can be written back to on-premises Exchange Server 2013 CU8 or later hybrid mode and appear as mail enabled distribution lists on premises. Does not require AAD Premium licences
  • Users can change their passwords via the login page or user settings in Office 365 and have that password written back online.
  • Exchange Server hybrid writeback is the classic writeback from Azure AD and is the apart from Group Writeback is the only one of these writebacks that does not require Azure AD Premium licences.
  • User writeback from Azure AD (i.e. users made in Office 365 in the cloud for example) to on-premises Active Directory
  • Password Hash Sync (this is not really writeback, but its the only permission needed by default for forward sync, so added here)
  • Windows 10 devices for “Azure AD Domain Join” functionality

All of these features require AADConnect and not and of the earlier verions. You can add all these writeback functions from the AADConect setup wizard, and if you have used Custom mode, then you will need to implement the following permissions.

In all the below sections you need to grant permission to the connector account. You can find the connector account for your Active Directory forest from the Synchronization Service program > Connectors > double-click your domain > select Connect to Active Directory Forest. The account listed here is the connector account you need to grant permissions to.

SourceAnchor Writeback

For users with (typically) multi-forest deployments or plans or a forest migration, the objectGuid value in Active Directory, which is used as the source for the attribute that keys your on-premises object to your synced cloud object – in AAD sync parlance, this is known as the SourceAnchor. If you set up AADConnect version 1.1.553.0 or later you can opt to change from objectGuid to a new source anchor attribute known as ms-ds-consistencyGuid. To be able to use this new feature you need the ability for AADConnect connector account to be able to read ObjectGUID and then write it back to ms-ds-consistencyGuid. The read permissions are typically available to the connector account without doing anything special, and if AADConnect is installed in Express Mode it will get the write permissions it needs, but as with the rest of this blog, if you are not using Express Mode you need to grant the permissions manually and so write permissions are needed to the ms-ds-consistencyGuid attribute. This can be done with this script.

$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number]. $ForestDN = "DC=contoso,DC=com" $cmd = "dsacls '$ForestDN' /I:S /G '`"$accountName`":WP;ms-ds-consistencyGuid;user'" Invoke-Expression $cmd | Out-Null 

$accountName = “domain\aad_account” #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number]. $ForestDN = “DC=contoso,DC=com” $cmd = “dsacls ‘$ForestDN’ /I:S /G ‘`”$accountName`”:WP;ms-ds-consistencyGuid;user'” Invoke-Expression $cmd | Out-Null [/code]

Note that if you use ms-ds-consistencyGuid then there are changes required on your ADFS deployment as well. The Issuance Transform Rules for the Office 365 Relying Party Trust contains a rule that specifies the ImmutableID (aka AADConnect SourceAnchor) that the user will be identified as for login. By default this is set to ObjectGUID, and if you use AADConnect to set up ADFS for you then the application will update the rule. But if you set up ADFS yourself then you need to update the rule.

Issuance Transform Rules

When Office 365 is configured to federate a domain (use ADFS for authentication of that domain and not Azure AD) then the following are the claims rules that exist out of the box need to be adjusted. This is to support the use of ms-ds-consistencyguid as the immutable ID.

ADFS Management UI > Trust Relationships > Relying Party Trusts

Select Microsoft Office 365 Identity Platform > click Edit Claim Rules

You get two or three rules listed here. You get three rules if you use -SupportMultipleDomain switch in Convert-MSOLDomainToFederated.
 
Rule 1:
Change objectGUID to ms-DS-ConsistencyGUID
 
Rule Was:
c:[Type == “http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname”]
=> issue(store = “Active Directory”, types = (“http://schemas.xmlsoap.org/claims/UPN”, “http://schemas.microsoft.com/LiveID/Federation/2008/05/ImmutableID”), query = “samAccountName={0};userPrincipalName,objectGUID;{1}”, param = regexreplace(c.Value, “(?<domain>[^\\]+)\\(?<user>.+)”, “${user}”), param = c.Value);
 
New Value:
c:[Type == “http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname”]
=> issue(store = “Active Directory”, types = (“http://schemas.xmlsoap.org/claims/UPN”, “http://schemas.microsoft.com/LiveID/Federation/2008/05/ImmutableID”), query = “samAccountName={0};userPrincipalName,ms-DS-ConsistencyGUID;{1}”, param = regexreplace(c.Value, “(?<domain>[^\\]+)\\(?<user>.+)”, “${user}”), param = c.Value);

Preparing for Device Writeback

Device Writeback is used where you have an on-premises ADFS server farm and implement Conditional Access on-premises. If you do cloud based authentication, including Pass-Through Auth (PTA) then you have no requirement for Device Writeback.

If you do not have a 2012 R2 or later domain controller then you need to update the schema of your forest. Do this by getting a Windows Server 2012 R2 ISO image and mounting it as a drive. Copy the support/adprep folder from this image or DVD to a 64 bit domain member in the same site as the Schema Master. Then run adprep /forestprep from an admin cmd prompt when logged in as a Schema Admin. The domain member needs to be a 64 bit domain joined machine for adprep.exe to run.

Wait for the schema changes to replicate around the network.

Import the cmdlets needed to configure your Active Directory for writeback by running Import-Module ‘C:\Program Files\Microsoft Azure Active Directory Connect\AdPrep\AdSyncPrep.psm1’ from an administrative PowerShell session. You need Azure AD Global Admin and Enterprise Admin permissions for Azure and local AD forest respectively. The cmdlets for this are obtained by running the Azure AD Connect tool.


$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number].
Initialize-ADSyncDeviceWriteBack -AdConnectorAccount $accountName -DomainName contoso.com #[domain where devices will be created].

In the domain partition you should now see an OU called RegisteredDevices. The AADSync account now has permissions to write objects to this container.

In Azure AD Connect, if you get the error “This feature is disabled because there is no eligible forest with appropriate permissions for device writeback” then you need to complete the steps in this section and click Previous in the AADConnect wizard to go back to the “Connect your directories” page and then you can click Next to return to the “Optional features” page. This time the Device Writeback option will not be greyed out.

Device Writeback needs a 2012 R2 or later AD FS server and WAP to make use of the device info in the Active Directory (for example, conditional access to resources based on the user and the device they are using). Once Device Writeback is prepared for with these cmdlets and the AADConnect Synchronization Options page is enabled for Device Writeback then the following will appear in Active Directory:

image

Not shown in the above, but adding the Display Name column in Active Directory Users and Computers tells you the device name. The registered owner and registered users of the device are available to view, but as they are SID values, they are not really readable.

Preparing for Group Writeback

Writing Office 365 “Modern Groups” back to Active Directory on-premises requires Exchange Server 2013 CU8 or later schema updates and servers installed. To create the OU and permissions required for Group Writeback you need to do the following.

Import the cmdlets needed to configure your Active Directory for writeback by running Import-Module ‘C:\Program Files\Microsoft Azure Active Directory Connect\AdPrep\AdSyncPrep.psm1’ from an administrative PowerShell session. You need Domain Admin permissions for the domain in the local AD forest that you will write back groups to. The cmdlets for this are obtained by running the Azure AD Connect tool.

$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number].
$cloudGroupOU = "OU=CloudGroups,DC=contoso,DC=com"
Initialize-ADSyncGroupWriteBack -AdConnectorAccount $accountName -GroupWriteBackContainerDN $cloudGroupOU

Once these cmdlets are run the AADSync account will have permissions to write objects to this OU. You can view the permissions in Active Directory Users and Computers for this OU if you enable Advanced mode in that program. There should be a permission entry for this account that is not inherited from the parent OU’s.

At the time of writing, the distribution list that is created on writeback from Azure AD will not appear in the Global Address List in Outlook etc. or allow on-premises mailboxes to send to these internal only cloud based groups. To add it to the address book you need to create a new subdomain, update public DNS and add send connectors to hybrid Exchange Server. This is all outlined in https://technet.microsoft.com/en-us/library/mt668829(v=exchg.150).aspx. This ensure’s that on-premises mailboxes can deliver to groups as internal senders and not require external senders enabled on the group. To add the group to the Global Address List you need to run Update-AddressList in Exchange Server. Once group writeback is prepared for using these cmdlets here and AADConnect has had it enabled during the Synchronization Options page, you should see the groups appearing in the selected OU as shown:

image

And you should find that on-premises users can send email to these groups as well.

Preparing for Password Writeback

The option for users to change their passwords in the cloud and have then written back to on-premises (with multifactor authentication and proof of right to change the password) is also available in Office 365 / Azure AD with the Premium Azure Active Directory or Enterprise Mobility Pack licence.

To enable password writeback for AADConnect you need to enable the Password Writeback option in AADConnect synchronization settings and then run the following three PowerShell cmdlets on the AADSync server:


Get-ADSyncConnector | fl name,AADPasswordResetConfiguration
Get-ADSyncAADPasswordResetConfiguration -Connector "contoso.onmicrosoft.com - AAD"
Set-ADSyncAADPasswordResetConfiguration -Connector "contoso.onmicrosoft.com - AAD" -Enable $true

The first of these cmdlets lists the ADSync connectors and the name and password reset state of the connector. You need the name of the AAD connector. The middle cmdlet tells you the state of password writeback on that connector and the last cmdlet enables it if needed. The name of the connector is required in these last two cmdlets.

To set the permissions on-premises for the passwords to be written back the following script is needed:

$passwordOU = "DC=contoso,DC=com" #[you can scope this down to a specific OU]
$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number].

$cmd = "dsacls.exe '$passwordOU' /I:S /G '`"$accountName`":CA;`"Reset Password`";user'"
Invoke-Expression $cmd | Out-Null

$cmd = "dsacls.exe '$passwordOU' /I:S /G '`"$accountName`":CA;`"Change Password`";user'"
Invoke-Expression $cmd | Out-Null

$cmd = "dsacls.exe '$passwordOU' /I:S /G '`"$accountName`":WP;lockoutTime;user'"
Invoke-Expression $cmd | Out-Null

$cmd = "dsacls.exe '$passwordOU' /I:S /G '`"$accountName`":WP;pwdLastSet;user'"
Invoke-Expression $cmd | Out-Null

Finally you need to run the above once per domain.

Preparing for Exchange Server Hybrid Writeback

Hybrid mode in Exchange Server requires the writing back on eight attributes from Azure AD to Active Directory. The list of attributes written back is found here. The following script will set these permissions for you in the OU you select (or as shown at the root of the domain). The DirSync tool used to do all this permissioning for you, but the AADSync tool does not. Therefore scripts such as this are required. This script sets lots of permissions on these eight attributes, but for clarify on running the script the output of the script is sent to Null. Remove the “| Out-Null” from the script to see the changes as they occur (the script also takes a lot longer to run).

$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number].
$HybridOU = "DC=contoso,DC=com"

#Object type: user
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;proxyAddresses;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchUCVoiceMailSettings;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchUserHoldPolicies;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchArchiveStatus;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchSafeSendersHash;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchBlockedSendersHash;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchSafeRecipientsHash;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msDS-ExternalDirectoryObjectID;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;publicDelegates;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchDelegateLinkList;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchDelegateListBL;user'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchDelegateListLink ;user'"
Invoke-Expression $cmd | Out-Null

#Object type: iNetOrgPerson
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;proxyAddresses;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchUCVoiceMailSettings;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchUserHoldPolicies;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchArchiveStatus;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchSafeSendersHash;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchBlockedSendersHash;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchSafeRecipientsHash;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msDS-ExternalDirectoryObjectID;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;publicDelegates;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchDelegateLinkList;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchDelegateListBL;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;msExchDelegateListLink;iNetOrgPerson'"
Invoke-Expression $cmd | Out-Null

#Object type: group
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;proxyAddresses;group'"
Invoke-Expression $cmd | Out-Null

#Object type: contact
$cmd = "dsacls '$HybridOU' /I:S /G '`"$accountName`":WP;proxyAddresses;contact'"
Invoke-Expression $cmd | Out-Null

Finally you need to run the above once per domain.

Preparing for User Writeback

[This functionality is not in the current builds of AADConnect]

Currently in preview at the time of writing, you are able to make users in Azure Active Directory (cloud users as Office 365 would call them) and write them back to on-premises Active Directory. The users password is not written back and so needs changing before the user can login on-premises.

To prepare the on-premises Active Directory to writeback user objects you need to run this script. This is contained in AdSyncPrep.psm1 and that is installed as part of Azure AD Connect. Azure AD Connect will install Azure AD Sync, which is needed to do the writeback. To load the AdSyncPrep.psm1 module into PowerShell run Import-Module ‘C:\Program Files\Microsoft Azure Active Directory Connect\AdPrep\AdSyncPrep.psm1’ from an administrative PowerShell session.

$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is an account usually in the form of AAD_number].
$cloudUserOU = "OU=CloudUsers,DC=contoso,DC=com"
Initialize-ADSyncUserWriteBack -AdConnectorAccount $accountName -UserWriteBackContainerDN $cloudUserOU

Once the next AADSync occurs you should see users in the OU used above that match the cloud users in Office 365 as shown:

image

Prepare for Password Hash Sync

This set of PowerShell ensures that the AADConnect account has the correct permissions to read password hashes from the Active Directory when they are changed, so that the service can sync them to the cloud. You need this permission whenever you enable Password Hash Sync (which could be in conjunction with another authentication method as well)

$DomainDN = "DC=contoso,DC=com"
$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number].

$cmd = "dsacls.exe '$DomainDN' /G '`"$accountName`":CA;`"Replicating Directory Changes`";'"
Invoke-Expression $cmd | Out-Null

$cmd = "dsacls.exe '$DomainDN' /G '`"$accountName`":CA;`"Replicating Directory Changes All`";'"
Invoke-Expression $cmd | Out-Null

Prepare for Windows 10 Registered Device Writeback Sync

Windows 10 devices that are joined to your domain can be written to Azure Active Directory as a registered device, and so conditional access rules on device ownership can be enforced. To do this you need to import the AdSyncPrep.psm1 module. This module supports the following two additional cmdlets to prepare your Active Directory for Windows 10 device sync:

CD "C:\Program Files\Microsoft Azure Active Directory Connect\AdPrep"
Import-Module .\AdSyncPrep.psm1
Initialize-ADSyncDomainJoinedComputerSync
Initialize-ADSyncNGCKeysWriteBack

These cmdlets are run as follows:

$accountName = "domain\aad_account" #[this is the account that will be used by Azure AD Connect Sync to manage objects in the directory, this is often an account in the form of MSOL_number or AAD_number].
$azureAdCreds = Get-Credential #[Azure Active Directory administrator account]

CD "C:\Program Files\Microsoft Azure Active Directory Connect\AdPrep"
Import-Module .\AdSyncPrep.psm1
Initialize-ADSyncDomainJoinedComputerSync -AdConnectorAccount $accountName -AzureADCredentials $azureAdCreds
Initialize-ADSyncNGCKeysWriteBack -AdConnectorAccount $accountName

To successfully run these cmdlets you need to have the latest version of the Microsoft Online PowerShell modules installed (the V1.1 versions, not the V2.0 preview). You can get these from https://www.powershellgallery.com/packages/MSOnline (which in turn needs MSOL Signin Assistant from https://www.microsoft.com/en-us/download/details.aspx?id=41950 and the Windows Management Framework v5 from https://www.microsoft.com/en-us/download/details.aspx?id=50395). If you get errors in the above, make sure you have the correct version, download from above and try the scripts again.

Once complete, open Active Directory Sites and Services and from the View menu Show Services Node. Then you should see the GUID of your domain under the Device Registration Configuration container.

image

Continuing Adventures in AD FS Claims Rules

Posted on 9 CommentsPosted in 2013, activesync, ADFS, ADFS 3.0, exchange online, https, Office 365, Outlook, OWA for Devices, Web Application Proxy

Updated 20th April 2017

There is an excellent article at http://blogs.technet.com/b/askds/archive/2012/06/26/an-adfs-claims-rules-adventure.aspx which discusses the use of Claims Rules in AD FS to limit some of the functionality of Office 365 to specific network locations, such as being only allowed to use Outlook when on the company LAN or VPN or to selected groups of users. That article’s four examples are excellent, but can be supplemented with the following scenarios:

  • OWA for Devices (i.e. OWA for the iPhone and OWA for Android), also known as Mobile OWA
  • Outlook Mobile Apps
  • Using Outlook on a VPN, but needing to set up the profile when also on the VPN
  • Outlook restrictions when using MAPI over HTTP
  • Legacy Auth and Modern Auth Considerations

OWA for Devices and AD FS Claims Rules

OWA for Devices is an app available from the Apple or Android store and provides mobile and offline access to your email, adding to the features available with ActiveSync. Though OWA for Devices is OWA, it also uses AutoDiscover to configure the app. Therefore if you have an AD FS claim rule in place the blocks AutoDiscover you will find that OWA for Devices just keeps prompting for authentication and never completes, though if you click Advanced and enter the server name by hand (outlook.office365.com) then it works.

Using OWA for iPhone/iPad diagnostics (described on Steve Goodman’s blog at http://www.stevieg.org/2013/08/troubleshooting-owa-for-iphone-and-ipad/) you might find the following entries in your mowa.log file:

[NetworkDispatcher exchangeURLConnectionFailed:withError:] , “Request failed”, “Error Domain=NSURLErrorDomain Code=-1012 “The operation couldn’t be completed. (NSURLErrorDomain error -1012.)” UserInfo=0x17dd2ca0 {NSErrorFailingURLKey=https://autodiscover-s.outlook.com/autodiscover/autodiscover.xml, NSErrorFailingURLStringKey=https://autodiscover-s.outlook.com/autodiscover/autodiscover.xml}”

And following the failure to reach https://autodiscover-s.outlook.com/autodiscover/autodiscover.xml successfully you might see the following:

[NetworkDispatcher exchangeURLConnectionFailed:withError:] , “Request failed”, “Error Domain=ExchangeURLConnectionError Code=2 “Timeout timer expired” UserInfo=0x17dc6eb0 {NSLocalizedDescription=Timeout timer expired}”

and the following:

[PAL] PalRequestManager.OnError(): ExchangeURLConnectionError2: Timeout timer expired

[autodiscover] Autodiscover search failed. Error: ExchangeURLConnectionError2: Timeout timer expired

[actions] Action (_o.$Yd 27) encountered an error during its execution: Error: Timeout timer expired

[autodiscover] TimerCallback_PeriodicCallback_HandleFailedAutodiscoverSearch took too long (XXXms) to complete

The reason is that the first set of AutoDiscover lookups work, as they are connecting to the on-premises endpoint and authentication for this endpoint does not go through AD FS. When you reach the end of the AutoDiscover redirect process you need to authenticate to Office 365, and that calls AD FS – and that might be impacted by AD FS claims rules.

The problem with OWA for Devices and claims rules can come about by the use of the AD FS Claims Rules Policy Builder wizard at http://gallery.technet.microsoft.com/office/Client-Access-Policy-30be8ae2. This tool can be adjusted quite easily for AD FS 2012 R2 (change line 218 to read “If (($OSVersion.Major –eq 6) –and ($OSVersion.Minor –ge 2))” (look for AD FS servers that are –ge [greater or equal to 2] rather than just 2). The AD FS Claims Rules Policy Builder has a setting called “Block only external Outlook clients” – and this blocks Outlook, Exchange Web Services, OAB and AutoDiscover from being used apart from on a network range that you provide. The problem is that you might want to block Outlook externally, but allow OWA for Devices to work.

To solve this problem make sure you do not block AutoDiscover in the claims rules. In the “Block only external Outlook clients” option, its the 2nd claim rule that needs to be deleted, so you end up with just the following:

image

With AutoDiscover allowed from all locations, but EWS and RPC blocked in Outlook, you still block Outlook apart from on your LAN, but you do not block OWA for Devices. Use exrca.com to test Outlook Autodiscover after making this change and you should find that the following error is not present when AutoDiscover gets to the autodiscover-s.outlook.com stage.

The Microsoft Connectivity Analyzer is attempting to retrieve an XML Autodiscover response from URL https://autodiscover-s.outlook.com/Autodiscover/Autodiscover.xml for user user@tenant.mail.onmicrosoft.com.

The Microsoft Connectivity Analyzer failed to obtain an Autodiscover XML response.

An HTTP 401 Unauthorized response was received from the remote Unknown server. This is usually the result of an incorrect username or password. If you are attempting to log onto an Office 365 service, ensure you are using your full User Principal Name (UPN).HTTP Response Headers:

If you changed an existing claims rule you either need to restart the AD FS service or wait a few hours for it to take effect.

OWA Apps and AD FS Claims Rules

The above section covers why MOWA (Mobile OWA) needs AutoDiscover to work. The same is true for any client application such as ActiveSync or Outlook for iOS / Outlook for Android / Windows Mail etc that needs to run AutoDiscover to determine your mailbox settings. Therefore, as in the above section ensure you are not blocking AutoDiscover from external networks unless apps on mobile devices are also only allowed to work on your WiFi.

Outlook with AD FS Claims Rules and VPN’s

As mentioned above, Outlook can be limited to working on a given set of IP ranges. If you block AutoDiscover from working apart from on these ranges (as was the issue causing OWA for Devices to fail above) then you can have issues with VPN connections.

Imagine the following scenario: Web traffic goes direct from the client, but Outlook traffic to Office 365 goes via the LAN. A split tunnel VPN scenario.

In this example, if AutoDiscover and initial profile configuration has never run you have a claims rule that allows Outlook to work on the VPN (as the public IP’s for your LAN/WAN are listed in the claims rules as valid source addresses), but AutoDiscover fails due to the same rules (as the web traffic is coming from the client and not the LAN). Therefore AutoDiscover fails and Outlook is never provisioned, so it appears as if Outlook is being blocked by the claim rules even though you are on the correct network for those claim rules.

Outlook with MAPI over HTTP

Updated April 20th 2017

A new connection protocol was released with Exchange Server 2013 SP1 called MAPI/HTTP and has been available in Exchange Online for a time before the release of it on-premises. After this protocol was released, support was made available for it from a range of client versions and currently it works with Outlook 2010 and later (as long as you are on the supported versions and not Outlook 2010 SP1 for example). There is no support for MAPI/HTTP on Outlook 2007 which is why this client will stop working with Office 365 in October 2017 (as Microsoft turn off RPC/HTTP at that time).

With this new protocol in place for Outlook 2010, it is important to ensure that the AD FS claim rule for x-ms-client-application with a value of Microsoft.Exchange.RPC is updated to include Microsoft.Exchange.Mapi and Microsoft.Exchange.Nspi. And come October 2017 when RPC/HTTP is turned off in Exchange Online, claim rules could be updated to remove Microsoft.Exchange.RPC from their checks.

Therefore if you are an Exchange Online user with claims rules for AD FS you need to add the following rules. These would have the same IP ranges as your Microsoft.Exchange.RPC rule and would be identical to this rule in every way, just the x-ms-client-application string needs to look for the following (one rule for each):

  • Microsoft.Exchange.Mapi
  • Microsoft.Exchange.Nspi

The Microsoft.Exchange.Mapi rule can be named “Outlook MAPI/HTTP” and the Nspi rule can be named “Outlook MAPI Address Book”. You need to keep the Exchange Web Services and Exchange Address Book rules in place, but once all users are migrated to MAPI/HTTP you can remove the Microsoft.Exchange.RPC rule.

Note that Exchange Online caches your AD FS credential’s for 24 hours for connections from a single IP address, so if you successfully connect to Exchange Online (say because you have not got the Microsoft.Exchange.Mapi block in place) then you will not connect back to AD FS for 24 hours and so not be affected by new rules that are added after you connect. If you want to test if your new Microsoft.Exchange.Mapi block rules work then you need to connect to Exchange Online via a different IP address, so a trip to the nearest coffee shop in the quest for network lockdown on your companies expense is called for!

Also it is possible to create a single rule rather than the multiple rules listed above. A single rule would contain the claim rule clause similar to the below:

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-client-application", Value =~ "Microsoft.Exchange.WebServices|Microsoft.Exchange.RPC|Microsoft.Exchange.Mapi|Microsoft.Exchange.Nspi"])

Note that this is different from what the Rule Builder wizard creates, as that creates a string comparison value, which is compared to the result with ==, but when more than one possible string is stored in the value the comparison operator needs to be =~ which is string compared to RegEx, as the | sign makes the value in the claim rule a RegEx expression.

Legacy Clients and Outlook 2016

Outlook 2016 supports modern authentication out of the box (unlike Outlook 2013 which required client side registry keys and certain hotfixes). But you still need to enable Modern Authentication on your tenant in Exchange Online.

To check if Modern Auth is enabled run Get-OrganizationConfig | FL *OAuth2* from a remote PowerShell session connected to Exchange Online. The value of OAuth2ClientProfileEnabled will be $True if Modern Authentication is enabled. If it is not enabled then Outlook 2016 will use Legacy authentication protocols and so will be blocked by the claim rules discussed here. If you enable Modern Auth though, Outlook 2010 is impacted by Claim Rules and Outlook 2013 June 2015 update + reg keys and Outlook 2016 and later are not impacted by the claim rules above (see below for these). Outlook with Modern Auth is restricted with the use of Conditional Access in Azure AD Premium. If you do not turn on Modern Auth you can use the legacy auth claim rule restrictions until the point where Microsoft enable Modern Auth for you – therefore it is best that if you are doing this then make your changes at your schedule and enable Modern Auth now, EMS to block Outlook 2016 and claim rules to block Outlook 2010 until the point where Microsoft removes support for that product.

Therefore a single claims rule for blocking legacy auth would look like the following:

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy"])
 && exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-client-application", Value =~ "Microsoft.Exchange.WebServices|Microsoft.Exchange.RPC|Microsoft.Exchange.Mapi|Microsoft.Exchange.Nspi"])
 && NOT exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-forwarded-client-ip", Value =~ "\b51\.141\.11\.170\b"])
 && exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/services/trust/2005/usernamemixed"])
 => issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

Note that in the above, though this comes from ADFS on Windows Server 2012 R2, we are still recommending the use of x-ms-proxy and x-ms-forwarded-client-ip. That is because the insidecorporatenetwork claim cannot be used for legacy authentication, as all auth request originate from Exchange Online (active auth) which is never on-premises! This claim rule is added to the Office 365 Relying Party Trust Issuance Authorization Rules after the default Permit Access to All Users rule that the Convert-MSOLDomainToFederated cmdlet or AADConnect configures. The claim rule permits access to all, then denies access if an legacy authentication session happens for Outlook protocols where the forwarded IP address is not the public IP for your LAN

Use Claim Rules to Block Modern Auth Clients

So the final scenario here, newly added to this old post in April 2017, requires issuing deny’s to claims when insidecorporatenetwork is false for the protocols/user agents that you want to block. For example the following will block all modern auth requests from outside the network from all applications apart from ActiveSync and AutoDiscover (as AutoDiscover is used by ActiveSync to set up the mobile device initially):

exists([Type == " http://schemas.microsoft.com/ws/2012/01/insidecorporatenetwork", Value == "false"])
 && NOT exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-client-application", Value =~ "Microsoft.Exchange.ActiveSync|Microsoft.Exchange.Autodiscover"])
 && exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/ls/"])
 => issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

Changing AD FS 3.0 Certificates

Posted on 7 CommentsPosted in 2012, 2012 R2, ADFS, ADFS 3.0, certificates, IAmMEC, Office 365, WAP, Web Application Proxy

I am quite adept at configuring certificates and changing them around, but this one took me completely by surprise as it has a bunch of oddities to consider.

First the errors: Web Application Proxy (WAP) reported 0x80075213. In the event log the following:

The federation server proxy could not establish a trust with the Federation Service.

Additional Data
Exception details:
The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

User Action
Ensure that the credentials being used to establish a trust between the federation server proxy and the Federation Service are valid and that the Federation Service can be reached.

And

Unable to retrieve proxy configuration data from the Federation Service.

Additional Data

Trust Certificate Thumbprint:
431116CCF0AA65BB5DCCD9DD3BEE3DFF33AA4DBB

Status Code:
Exception details:
System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.IdentityServer.Management.Proxy.StsConfigurationProvider.GetStsProxyConfiguration()

This was ultimately caused by the certificate on the AD FS Server having been replaced in the user interface, but this did not replace the certificate that HTTP was using or the published web applications and the certificates they were using.

So here are the steps to fix this issue:

AD FS Server

  1. Ensure the certificate is installed in the computer store of all the AD FS servers in the farm
  2. Grant permissions to the digital certificate to the ADFS Service account. Do this by right-clicking the new digital certificate in the MMC snap-in for certificates and choosing All Tasks > Manage Private Keys. Grant read permission to the service account that ADFS is using (you need to click Object Types and select Service Accounts to be able to select this user). Repeat for each server in the farm.
  3. In the AD FS management console expand service > certificates and ensure that the service communications certificate is correct and that the date is valid.
  4. Open this certificate by double-clicking it and on the Details tab, check the value for the Thumbprint.
  5. Start PowerShell on the AD FS Server and run Get-AdfsSslCertificate (not Get-AdfsCertificate). You should get back a few rows of data listing localhost and your federation service name along with a PortNumber and CertificateHash. Make sure the CertificateHash matches the Thumbprint for the service communications certificate.
  6. If they do not, then run Set-AdfsSslCertificate -Thumbprint XXXXXX (where XXXXX is the thumbprint value without spaces).
  7. Then you need to restart the ADFS Server.
  8. Repeat for each member of the farm, taking them out and in from any load balancer configuration you have. Ensure any SSL certs on the load balancer are updated as well.

Web Application Proxy

  1. Ensure the certificate is installed in the computer store of the web application proxy server as well. Permissions do not need to be set for this service.
  2. Run Get-WebApplicationProxySslCertificate. You should get back a few rows of data listing localhost and your federation service name along with a PortNumber and CertificateHash. Make sure the CertificateHash matches the Thumbprint for the service communications certificate. If it does not use Set-WebApplicationProxySslCertificate -Thumbprint XXXXX to change it. Here XXXXX is the thumbprint value without spaces.
  3. Restart WAP server and it should now connect to the AD FS endpoint and everything should be working again.
  4. Check that each Web Proxy Application is using the new certificate. Do this with Get-WebApplicationProxyApplication | Set-WebApplicationProxyApplication -ExternalCertificateThumbprint XXXXXX (where XXXXX is the thumbprint value without spaces). This sets all your applications to use the same certificate. If you have different certificates in use, then do them one by one with Get-WebApplicationProxyApplication | Format-Table Name,ExternalCert* to see what the existing thumbprints are and then Get-WebApplicationProxyApplication “name” | Set-WebApplicationProxyApplication -ExternalCertificateThumbprint XXXXXX to do just the one called “name”.