AutoDetect is like an “AutoDiscover of AutoDiscovers”. Exchange Online and Exchange Server administrators know the delights of AutoDiscover, which is a web service to tell the client where the mailbox is located (to return the endpoints and protocols that email is read from by the client). But AutoDiscover only works for Exchange based services (unless you hack the autodiscover.xml file, and more on that later) so what about clients like Outlook that can connect to many different email services (i.e. Gmail, IMAP servers, and of course the Microsoft provided ones).
Historically the Outlook desktop product has a feature called GuessSmart to detect your email settings if you are not using Exchange (i.e. Gmail, Yahoo etc.). But Outlook Mobile and Outlook desktop use a cloud service called AutoDetect. Originally this was only used by Outlook Mobile when it was a seperate product called Acompli but it is now used instead of GuessSmart in the desktop versions of Outlook as well.
Microsoft host AutoDetect at https://prod-autodetect.outlookmobile.com/detect and it is on the IPs 52.125.128.0/20 and 52.127.96.0/23 (at the time of writing, see Row 10 here) and it is the job of AutoDetect, as described above, to work out where your email endpoints are. But it can get this wrong! And when it goes wrong, the client has difficulty connecting becuase (maybe) one set of endpoints are returned to the client, but your email is on a different endpoint.
This can be an issue with a migration, and apparently you can raise a Microsoft Support ticket to have the AutoDetect cache cleared, but this blog post aims to try to describe why AutoDetect made the decision it made in the first place.
This is not using any information I got from Microsoft and is all a guess – but maybe an educated guess!!
Exchange Online and Exchange Server
If your mailbox is located on either of these platforms then AutoDetect will find your mailbox via AutoDiscover v1 or v2, preferencing AutoDiscover v2. AutoDiscover v2 is located at https://autodiscover-s.outlook.com/autodiscover/autodiscover.json?Email=<email-address> or the same path to your Exchange Server 2016 or later server at (typically) https://autodiscover.domain.com but can be at other FQDNs. AutoDiscover v1 is at the same location, but the webservice returns XML for v1 and JSON for v2 (notice the difference in the path in the URLs above).
So if the JSON endpoint is not reachable (for example blocked by a firewall) then AutoDiscover v2 is not available.
AutoDetect will use AutoDiscover v2 or v1, in that order, to retrieve the endpoints to your service. Where you can have issues here is during migrations (for example) where the mailbox has yet to move to the cloud but AutoDiscover v2 is returning Exchange Online as a valid endpoint. This happens when you have a mailbox in Exchange Online but also a mailbox somewhere on-premises or another cloud provider and you are migrating data into it, but have not moved the user across yet.
This service can also get the wrong endpoint when AutoDiscover v1 is pointing to Exchange Online when you still have mailboxes in Exchange Server. AutoDiscover needs to point to Exchange Server whilst you have mailboxes in that location and Exchange Server redirects AutoDiscover to the cloud. The other direction does not work.
Gmail
AutoDetect will use your MX records instead of AutoDiscover to determine that your domain is in Gmail. If you have an AutoDiscover DNS record for your domain that works though, that will be prioritised over fixed domain decisions like MX records.
The AutoDetect decision process is as follows: “autoDv2 > autoDv1 > fixed db provider > fixed db domain protocols > db provider > db domain protocols” and Gmail falls into the third category, “fixed db provider”. AutoDetect determines that the mailbox is on Google and returns protocol endpoints for IMAP, POP and SMTP. In the case of IMAP it is imap.gmail.com on port 993 using SSL based encryption.
I am not sure what is meant by “fixed db domain protocols” etc. in the above priority order, and if I find out I will update this blog post! This is one of the bits of this article I am making educated guesses about!
The AutoDetect service could therefore get the wrong endpoint if you are migrating from Gmail to Exchange Online and have created an AutoDiscover endpoint with an active mailbox licenced (AutoDetect will not return data from Exchange Online where there is just a user account or a mailuser/contact/guess account object – only a mailbox. Therefore, for migrations and the pre-copy of data – it can be a good decision to give the mailbox an alternative email address for the pre-copy stage of the migration. For example, you are migrating bernt@domain.com from Gmail, so migrate it to bernt@domain.onmicrosoft.com and only cutover the email address when you cutover the user as well. This benefits more than just AutoDetect and is helpful in staged migrations as well.
Yahoo and Apple
AutoDetect can return endpoint data for ymail.com and Apple Mail services. This can be seen in the URL used by the clients to contact AutoDetect. The URL is:
And the URL contains the various services it supports. We will see more of this in the debug section below.
IMAP/POP Servers
As mentioned above, AutoDetect replaces Guessmart in the Outlook client to determine the endpoint. To do this, AutoDetect will have some email providers already in their system (typically when using their domain name) or make connections to service endpoints such as imap.domain.com and pop.domain.com to see if they work. These are “fixed db provider” or “db domain protocols” in the processing order above. For example some of the larger providers are known to the service and so the correct endpoints are returned when you are using their domain (for example zoho.com).
Where your email provider is not one of the larger ISPs or you are using your own domain, the AutoDetect determination is more guessy – and this is where service endpoints such as imap.domain.com can be tried against and if successful added to the AutoDetect response. But, AutoDetect will try all the steps – and this means it will try AutoDiscover v1 if you place an xml file at /autodiscover/autodiscover.xml on autodiscover.domain.com that has a valid certificate and responds within a short amount of time to a POST request. Where I have seen AutoDetect report that it cannot do AutoDiscover, but the AutoDiscover endpoint exists (even where it is statically set to a fixed xml file), in in cases where over approx. 10 seconds are needed to query the XML file via the POST method. This is considered invalid XML (i.e. errors like “The Response element in the payload was null”). This can be seen in tools like exrca.com which is used for testing AutoDiscover and other endpoints.
AutoDetect will also cache responses, and especially if previous attempts to determine the server endpoints worked, but more recent attempts are failing. I have previously written about intermediate certificates not being published on load balancers breaking AutoDetect but if the AutoDiscover endpoint for your newly migrated endpoint is not fast enough, or contains invalid XML, or the TLS protocol is too old or the ciphers are not the same as what Microsoft will connect to, then the cache will be returned and not your new server endpoint settings. Microsoft Support could probably clear the cache in this case, but if your new endpoint is “too old” from a security prespective then the AutoDetect will not update to the newer correct information either.
For example, I have seen AutoDiscover endpoints operating over TLS 1.2 only (good), but with very few ciphers that Microsoft Exchange Remote Connectivity Analyzer supports, and I know from experience that AutoDetect supports even less ciphers. So if you get a low cipher count error in exrca.com for your AutoDiscover test (or not supporting TLS 1.2) then update your autodiscover hosting server to support more up to date encryption ciphers and protocols and see if AutoDetect returns correct information.
Debug
So now we see what AutoDetect can do, lets see how to debug it. There are two ways to do this, one being to use an API query tool (I’ve used this in the past) and the other being PowerShell. We will look at PowerShell Invoke-WebRequest here.
As mentioned above, AutoDetect is at https://prod-autodetect.outlookmobile.com/detect and the following PowerShell will query this endpoint for the provided email address (on line 1) and return both the discovered information on the mailbox location, and the AutoDetect debug information I have referenced above:
$email = "brian@gmail.com"
$AutoDetect = Invoke-WebRequest -Uri 'https://prod-autodetect.outlookmobile.com/detect?services=office365,outlook,google,icloud,yahoo&protocols=rest-cloud,rest-outlook,rest-office365,eas,imap,smtp' -Headers @{'x-email'=$email}
$Results = ConvertFrom-Json $AutoDetect.Content
$debug = $AutoDetect.Headers.'x-debug-support'
$AutoDv2Error = $AutoDetect.Headers.'X-AutoDv2-Error'
Write-Host "Results/Services and Protocols" -ForegroundColor Green
Write-Host "Service Used" -ForegroundColor Green
$Results.Services.service
Write-Host "Protocols and Endpoints" -ForegroundColor Green
$Results.Protocols
Write-Host "Debug Info" -ForegroundColor Red
[System.Text.Encoding]::ASCII.GetString([System.Convert]::FromBase64String($debug))
Write-Host "AutoDV2 Errors" -ForegroundColor Red
$AutoDv2Error
The “debug” info (encoded in Base64) returns what AutoDetect has detected but not always why it has made that decision, and this is the black box that is AutoDetect. The above educated guesses might help you work out what is going on given the results that PowerShell returns for you.
The AutoDV2 Errors in the above report will tell you if AutoDiscover V2 is broken (applicable if your mailbox is Exchange Online or Exchange Server with Hybrid Modern Authentication enabled). If you get an error about certificates against “AutoDV2 Errors” then take a look at this blog post from me in 2021.
Photo by Nubia Navarro (nubikini) from Pexels
Leave a Reply