Exchange Server and Missing Root Certificates

Posted on Leave a commentPosted in 2007, 2010, 2013, exchange, exchange online, Exchange Server, federation, Free/Busy

I came across an issue with a clients Exchange Server deployment today that is not well documented – or rather it is, but you need to know where to look. So I thought I would document the troubleshooting steps and the fix here.

We specifically came across this error when testing Free/Busy for an Office 365 migration, though it could happen for a variety of reasons. Free/Busy and other lookups in a cross-forest Exchange Server deployment require a working organization configuration and this was failing. Running Test-FederationTrust (a prerequisite of the organization relationship) in verbose mode (add -Verbose to the end) returned the following:

Unable to retrieve federation metadata from the security token
service. Reason: Microsoft.Exchange.Management.FederationProvisioning.FederationMetadataException: Unable to access the
Federation Metadata document from the federation partner. Detailed information: “The underlying connection was closed:
Could not establish trust relationship for the SSL/TLS secure channel.”.

The final result of the test will also show two errors for “Unable to retrieve federation metadata from the security token service.” and “Failed to request delegation token.”

The last part of the verbose error is the clue here. The server in question is unable to make an SSL/TLS connection to the endpoint that the federation trust needs to reach to get the federation trust metadata. That endpoint is listed right at the start of the Verbose output. It reads:

VERBOSE: [16:53:08.306 GMT] Test-FederationTrust : Requesting Federation Metadata from
https://nexus.microsoftonline-p.com/FederationMetadata/2006-12/FederationMetadata.xml.

Now that we have a URL and an error message, check that the URL is reachable from each of your Exchange Servers. At my client today we found one server could not successfully reach this endpoint without an SSL error turning up in the browser. The problem was that the certificate that the endpoint is secure with is issued by the Baltimore Cybertrust Root Certificate – one that Microsoft uses for lots of services, but the root certificate was not installed on that machine. Lots of root certs where missing from that machine as it had never had a root certificate update applied to it.

We installed the latest Root Certificate Update and then the federation trust worked and free/busy etc. (mail tips, cross-forest message tracking etc.) all worked fine.

Removing Edge Subscription When Exchange 2013 Installed

Posted on 6 CommentsPosted in 2007, 2010, 2013, Edge, exchange, federation, IAmMEC, mcm, mcsm, smarthost, transport

Exchange 2013 does not have an Edge role (at the time of writing – Aug 2013). It is possible to use Exchange 2010 SP3 and install the Edge role should you need one.

There is a problem though when it comes to removing the Edge Subscription between an organization that contains Exchange 2013 servers and the Exchange 2010 Edge Server. To remove the subscription on the Edge server role you run Remove-EdgeSubscription servername and this removes both the subscription and any subscribed objects from the local AD LDS database on that Edge Server. But if any of these subscribed objects where created on Exchange 2013 after it was installed, then they will have an ExchangeVersion equal to 0.20 (15.0.0.0). The Exchange 2010 SP3 Remove-EdgeSubscription cannot process this object and so fails with:

Remove-EdgeSubscription : Can’t make this change because the object’s ExchangeVersion property is 0.20 (15.0.0.0), which is not supported by the current version 0.1 (8.0.535.0). You will need a newer version of Exchange to make this change. Property Name: ExchangeVersion
At line:1 char:24
+ Remove-EdgeSubscription <<<<  edge2 -Verbose
+ CategoryInfo          : NotSpecified: (0:Int32) [Remove-EdgeSubscription], DataValidationException
+ FullyQualifiedErrorId : D8A49A14,Microsoft.Exchange.Management.SystemConfigurationTasks.RemoveEdgeSubscription

The way to fix this is to find and manually remove the object with an ExchangeVersion of 0.20 (15.0.0.0) from the AD LDS database and then repeat the Remove-EdgeSubscription cmdlet – as that should now work (unless you have two or more objects with the higher version number to locate and delete).

  1. To find objects with an ExchangeVersion greater than “0.1 (8.0.535.0)”, which is the version Exchange 2010 will process, open ADSIEdit on the Edge server.
  2. Right-click the ADSI Edit node at the top of the window and choose Connect to…
    image
  3. In the Connection Settings dialog (shown above), change the Select a well known Naming Context to Configuration and type the local server name and the AD LDS port in the Select or type a domain or server field. The server:port value should be EDGESERVERNAME:50389
    image
  4. Expand the tree view until you reach CN=First Organization,CN=Microsoft Exchange,CN=Services,CN=Configuration,CN={AD258B4D-CCB4-4125-80C1-7B73CE066341}
    image
  5. You now need to look at each object, starting with those you remember having created since 2013 was installed, for an object who’s msExchVersion value is not 4535486012416. For example, in the below screenshot I have an accepted domain made due to Federation in Exchange 2013. This object (under CN=Accepted Domains,CN=Transport Settings,…) has a value of 88218628259840.
    image
  6. To validate that this is the correct object to manually delete, from the Exchange Management Shell on the Edge server, enter Get-Object | FT name,ExchangeVersion where Object is the cmdlet that you are looking to query – in my case it would be Get-AcceptedDomain.
    image
  7. As you can see, this object has a newer ExchangeVersion and so it is (at least) this object that is stopping Exchange EdgeSync from being removed.
  8. Manually delete this object in ADSI Edit (it is safe to do this as it will resync from the Exchange organization if you recreate Edge Subscription later). Do not delete it from the Active Directory with ADSI Edit – just from AD LDS. Take care to only delete this object and not the parent object.
  9. Once this object is gone, try Remove-EdgeSubscription servername again. If this is the only object, then the Edge Subscription will be removed in the Edge Server

You can now carry on with whatever it was that you were doing that required a removal of the Edge Subscription.

Fix Federation Trust Issues After Exchange Server Recovery

Posted on 1 CommentPosted in 2010, exchange, exchange online, federation

I had a need to recover an Exchange Server following a blue screen after patching that I could not undo. I had the databases intact, and of course Active Directory was installed on a different server so I installed a new server and ran setup /m:recoverserver (after installing all the updates and hotfixes that is).

Upon completion and remounting of the databases everything worked fine apart from some errors in the event log about federation trust certificates being missing. And of course, I did not have these in my certificate backup!

In Exchange 2010 RTM federation trust certificates needed to be publically issued certs, but from SP1 and onwards they can be self created by your Exchange Server and here is where the problem lies – because the certificates are self issued I never went through the process of ensuring I had an independent backup of them. Therefore I could not remove them or change them in Exchange Server.

First I was getting the following event log error:

Federation Certificate Not Found: thumbprint_value. Unable to find the certificate in the local or neighboring sites. Confirm that the certificate is available in your topology and if necessary, reset the certificate on the Federation Trust to a valid certificate using Set-FederationTrust.  The certificate may take time to propagate to the local or neighboring sites.

Attempts to Get-FederationTrust or Set-FederationTrust failed, presumable becuase I do not have the correct certificate installed.

Remove-FederationTrust fails because it is in use by some listed organizations, so I tried various other options. In summary it was impossible to remove the federation trust nor was it possible to create a new federation certifcate and move over to it. If I had multiple Exchange Servers in this organization then the certificate would have been retrieved from another server – but this is a single server organization.

So I resorted to removing the federation trust directly from ADSI Edit with the intention of creating a new one immediately and then removing and recreating that one straight away to attempt to clean it all up correctly.

The object to remove is CN=Microsoft Federation Gateway,CN=Federation Trusts,CN=OrgName,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=domain

This allowed me to create a new federation trust, though I did need to go through the domain proof steps again.

OWA and Moving Mailboxes to Office 365

Posted on Leave a commentPosted in 2010, ADFS, ADFS 2.0, certificates, exchange, exchange online, federation, Office 365, organization relationships, owa, powershell

Lets imagine a scenario where you are using an on-premises Exchange Server and users’ use Outlook Web App, and then you move some mailboxes to the Office 365 cloud with Hybrid Coexistence enabled. The user might not know their mailbox has been moved and so yesterday they went to https://mail.company.com/owa, but today they need to visit https://outlook.com/owa/company.com (where company.com is the domain name in your login name).
But becuase the user does not know that their mailbox has been moved when they visit https://mail.company.com/owa they get an error that their OWA URL is out of date.
To fix this, and provide the user with the correct URL (https://outlook.com/owa/company.com) then you need to set the TargetOwaURL property of the Organization Relationship that you have configured for your Office 365 service domain.

Set-OrganizationRelationship name -TargetOwaURL https://outlook.com/owa/company.com

Now when users login with an account that has been moved to the cloud they will be told that their mailbox has moved, and that they should visit https://outlook.com/owa/company.com.
Some organizations though have an issue with this URL – it does not mention the company name in the domain name bit, and a name such as http://webmail.company.com/owa would be preferred for mailboxes moved to the cloud. To present the user with this URL after they login to on-premises OWA or for a URL that you can just tell them to use you need to do two things:

  1. Create a CNAME record in DNS for webmail that has outlook.com as the target FQDN. The CNAME record can be anything that is not already in use for the domain (for example it could be mail if that is not in use).
  2. The TargetOwaURL property of the Organization Relationship needs to be http://webmail.company.com/owa. The TargetOwaURL must finish with /owa or the on-premises OWA redirect page will error and the domain name used must be the domain name in your login name.

The outlook.com server will take the CNAME value provided by the browser and do realm discovery on this name – that is it will redirect you to the correct login server for your domain.

Assign Specific Licences in Office 365 Via PowerShell

Posted on 18 CommentsPosted in 2010, exchange, exchange online, federation, licence, mcm, Office 365

 

To add specific licences to users in Office 365 without using the portal, and to assign subsets of the licences available requires two things. First you need to enumerate the licences and licence service plans, then you need to assign the new plan you have created to your users. This can be performed in bulk and is repeatable unlike when using the portal.
First, enumerate the licence plans and create your own licence:

  1. Open Microsoft Online Services Module for Windows PowerShell and connect to the service
    • $cred = Get-Credential
    • Connect-MsolService -Credential $cred
  2. Get-MsolAccountSku | Format-Table AccountSkuId, SkuPartNumber
    • The second column in this list is referenced in the next command as [SkuPartNumber]
  3. $ServicePlans = Get-MsolAccountSku | Where {$_.SkuPartNumber -eq “[SkuPartNumber]”}
  4. $ServicePlans.ServiceStatus
    • This returns all the service plans
  5. $MyO365Sku = New-MsolLicenseOptions -AccountSkuId [tenantname:AccountSkuId] -DisabledPlans Comma_Seperated_List_From_ServicePlans_Output

Secondly you need to assign the licence to the user(s):

  1. Set-MsolUser -UserPrincipalName user@domain.com -UsageLocation GB
  2. Set-MsolUserLicense -UserPrincipalName user@domain.com -AddLicenses [tenantname:AccountSkuId] -LicenseOptions $MyO365Sku
  3. Repeat for any other licences you want to apply for other users or other licence options you want to apply to this user.

For reference, the SkuPartNumber’s that we discovered are:

Inside ENTERPRISEPREMIUM_NOPSTNCONF (E5 without PSTN Conferencing) Sku:

  • EQUIVIO_ANALYTICS
  • LOCKBOX_ENTERPRISE
  • EXCHANGE_ANALYTICS
  • SWAY
  • ATP_ENTERPRISE
  • MCOEV
  • BI_AZURE_P2
  • INTUNE_O365
  • PROJECTWORKMANAGEMENT
  • RMS_S_ENTERPRISE
  • YAMMER_ENTERPRISE
  • OFFICESUBSCRIPTION
  • MCOSTANDARD
  • EXCHANGE_S_ENTERPRISE
  • SHAREPOINTENTERPRISE
  • SHAREPOINTWAC

Inside ENTERPRISEPREMIUM (E5) Sku:

  • EQUIVIO_ANALYTICS
  • LOCKBOX_ENTERPRISE
  • EXCHANGE_ANALYTICS
  • SWAY
  • ATP_ENTERPRISE
  • MCOEV
  • BI_AZURE_P2
  • INTUNE_O365
  • PROJECTWORKMANAGEMENT
  • RMS_S_ENTERPRISE
  • YAMMER_ENTERPRISE
  • OFFICESUBSCRIPTION
  • MCOSTANDARD
  • EXCHANGE_S_ENTERPRISE
  • SHAREPOINTENTERPRISE
  • SHAREPOINTWAC
  • MCOMEETADV (PSTN Conferencing)

Inside ENTERPRISEPACK (E3) Sku:

  • YAMMER_ENTERPRISE (Yammer – though you cannot apply this individually or disable it individually, so ignore it for the purposes of this script)
  • OFFICESUBSCRIPTION (this is Office Professional Plus)
  • MCOSTANDARD (this is Lync)
  • SHAREPOINTWAC (this is Office Web Apps)
  • SHAREPOINTENTERPRISE
  • EXCHANGE_S_ENTERPRISE (Exchange Plan 2)
  • RMS_S_ENTERPRISE (Azure Rights Management)
  • INTUNE_O365 (Mobile Device Management for Office 365)
  • SWAY

Inside Enterprise Mobility Pack (EMS)

  • RMS_S_PREMIUM
  • INTUNE_A
  • RMS_S_ENTERPRISE
  • AAD_PREMIUM
  • MFA_PREMIUM

Inside ENTERPRISEPACKWITHOUTPROPLUS sku

  • YAMMER_ENTERPRISE
  • SHAREPOINTWAC
  • SHAREPOINTENTERPRISE
  • RMS_S_ENTERPRISE
  • MCOSTANDARD
  • EXCHANGE_S_ENTERPRISE
  • INTUNE_O365

Inside DESKLESSWOFFPACK Sku:

  • SHAREPOINTWAC
  • SHAREPOINTDESKLESS
  • EXCHANGE_S_DESKLESS

Inside EXCHANGESTANDARD sku

  • INTUNE_O365
  • EXCHANGE_S_STANDARD

Inside EXCHANGEENTERPRISE sku

  • INTUNE_O365
  • EXCHANGE_S_ENTERPRISE

Inside EXCHANGEARCHIVE Sku

  • EXCHANGE_S_ARCHIVE

Inside P1 (Small Business) Tenants

  • MCOLITE
  • SHAREPOINTLITE
  • EXCHANGE_L_STANDARD

Inside K1 – DESKLESSPACK

  • SHAREPOINTDESKLESS
  • EXCHANGE_S_DESKLESS

Inside K2 – DESKLESSWOFFPACK

  • SHAREPOINTWAC
  • SHAREPOINTDESKLESS
  • EXCHANGE_S_DESKLESS

Inside P1 – LITEPACK

  • MCOLITE
  • SHAREPOINTLITE
  • EXCHANGE_L_STANDARD

Inside E1 – STANDARDPACK

  • MCOSTANDARD
  • SHAREPOINTSTANDARD
  • EXCHANGE_S_STANDARD

Inside E4 – ENTERPRISEWITHSCAL

  • YAMMER_ENTERPRISE
  • OFFICESUBSCRIPTION
  • MCOSTANDARD
  • SHAREPOINTWAC
  • SHAREPOINTENTERPRISE
  • EXCHANGE_S_ENTERPRISE
  • RMS_S_ENTERPRISE

Inside PowerBI Standalone (POWER_BI_STANDALONE)

  • YAMMER_ENTERPRISE
  • SQL_IS_SSIM
  • BI_AZURE_P1
  • SHAREPOINTENTERPRISE

Inside Project Online (PROJECTONLINE_PLAN_1)

  • SWAY
  • SHAREPOINT_PROJECT
  • SHAREPOINTWAC
  • SHAREPOINTENTERPRISE

Inside Project Lite (PROJECTESSENTIALS)

  • SWAY
  • SHAREPOINTWAC
  • SHAREPOINTENTERPRISE
  • PROJECT_ESSENTIALS

Inside Academic A2 Plans

  • SHAREPOINTWAC_EDU
  • MCOSTANDARD
  • SHAREPOINTSTANDARD_EDU
  • EXCHANGE_S_STANDARD

Inside Medium Business Sku (contoso:MIDSIZEPACK)

  • SHAREPOINTWAC
  • OFFICESUBSCRIPTION
  • EXCHANGE_S_STANDARD_MIDMARKET
  • SHAREPOINTENTERPRISE_MIDMARKET
  • MCOSTANDARD_MIDMARKET

PowerBI Standard (POWER_BI_STANDARD)

  • BI_AZURE_P0

Visio Pro for Office 365

  • VISIOCLIENT

Project Pro for Office 365

  • PROJECTCLIENT

Skype for Business PSTN Conferencing

  • MCOMEETADV

Skype for Business PSTN Domestic and International Calling

  • MCOPSTN2

Skype for Business Cloud PBX

  • MCOEV

Microsoft Dynamics CRM Online internal use rights (IUR) benefit for MPN members

  • CRMIUR

With thanks to Donte Henry (Avanade) and Tim Heeney (Microsoft). Discovered during the Office 365 MCM Class for Exchange 2010 MCM’s.

Updated June 2014 with the findings of some of those who added comments below. Note that some comments say you need to have an array for disabled plans – this is not what I find when I run the above.

Updated Feb 2015 with more licence pack data.

Updated June 2015 with more licence pack data (INTUNE_O365)

Updated Dec 2015 with E5/NOPSTN and new standalone licence skus

Updated Feb 2016 with E5 and PSTN Conferencing

Updated May 2016 with Skype for Business Cloud PBX and some Dynamics CRM

Free/Busy Cross-Forest Working One Way Only

Posted on Leave a commentPosted in 2010, exchange, federation, Office 365, organization relationships, powershell, proxy

Or indeed, not working at all! I had the issue of it working one way only (On-Premise Exchange organization > Office 365) but the other way (cloud to on-premise) did not work at all.

The answer is shown in this video

http://www.microsoft.com/showcase/en/us/details/a16a9d39-416a-4b01-a88f-5ff511580424

This covers the reasons why Free/Busy (and the other federation features of MailTips, archive and move mailbox might not work both ways in a Hybrid Coexistence setup for Office 365 or between two Exchange on-premise organizations.

The reason I found was the Organization Relationship contained the wrong list of domains. There are three domains (at least) that are needed in the organization relationship. These are:

  • Primary SMTP Namespace Domain (i.e. fabrikam.com)
  • Namespace for other organization (i.e. service.fabrikam.com)
  • Exchange Delegation domain (i.e. exchangedelegation.fabrikam.com)

In the organization relationship on-premise (or Org A if you are doing two on-premise organizations) set the following domains after the relationship is created. This includes the primary SMTP namespace and the service namespace for the other organization. This can be set with the following Exchange Management Shell cmdlet:

Set-OrganizationRelationship -Identity “To Cloud” -DomainNames “service.fabrikam.com”,”fabrikam.com” -MailTipsAccessEnabled $True -MailTipsAccessLevel All -DeliveryReportEnabled $True –TargetOwaUrl https://outlook.com/owa/fabrikam.com -ArchiveAccessEnabled $True –MailboxMoveEnabled $True

In Org B (or on Office 365) use a similar cmdlet, but use the Exchange Delegation namespace and the primary SMTP domain. Also Office 365 does not let you set the MailboxMoveEnabled property to $True

Set-OrganizationRelationship -Identity “To On-premises” -DomainNames “exchangedelegation.fabrikam.com”,”fabrikam.com” -MailTipsAccessEnabled $True -MailTipsAccessLevel All -DeliveryReportEnabled $True -ArchiveAccessEnabled $True

Supposedly Service Pack 2 for Exchange 2010 will do all this and more for you with the Hybrid Configuration Wizard, but its always useful for troubleshooting to discover what changes and why when you run a wizard to do things!