Getting Exchange 2010 SP1 Diagnostics

Posted on Leave a commentPosted in 2010, active directory, exchange

New with Exchange Server 2010 SP1 is the Get-ExchangeDiagnosticInfo PowerShell cmdlet. This is not documented anywhere online, so I thought I would start a trend!

Get-ExchangeDiagnosticInfo reports information on the information and status of Exchange Server as seen by individual processes. The information returned is in the form of a blob of XML data and in my next blog I will show how to process that information into a more readable form.

At the time of writing Get-ExchangeDiagnosticInfo only reports information on the Mailbox Server role and the Hub Transport role, and only for the sending and receiving of email – so its currently an exclusive cmdlet for Exchange Transport.

From an Exchange Management Shell window, start with just Get-ExchangeDiagnosticInfo to get a list of processes on the machine that can be queried. The Result value reports something like this:

<Diagnostics>
<ProcessLocator>
<count>2count>
<Process>
<guid>guid>
<id>2356id>
<name>MSExchangeMailSubmissionname>
Process>
<Process>
<guid>guid>
<id>3408id>
<name>EdgeTransportname>
Process>
ProcessLocator>
Diagnostics>

Of most interest from this output are the two processes that you can get diagnostics from. These two are the Mail Submission Service and the Edge Transport process. Mail Submission’s job is to notify any Hub Transport role in the same Active Directory site that it has email waiting to be collected and the EdgeTransport.exe process does all the work of the collecting, processing and delivering emails onward.

To report further information on these processes use the Process parameter with Get-ExchangeDiagnosticInfo

Get-ExchangeDiagnosticInfo –Process EdgeTransport

or

Get-ExchangeDiagnosticInfo –Process MSExchangeMailSubmission

For this level of reporting you get even more XML returned, but the interesting XML data to look at now is the Components group. EdgeTransport returns TransportConfiguration, ResourceManager, TransportDumpster, RmsClientManager, ShadowRedundancy, SmtpOut and StoreDriver. The MSExchangeMailSubmission process allows the querying of the MailSubmission and RedundancyManager.

Both of these two cmdlets above also return the “Supported arguments” data (part of the Help XML blob). These values can be used on the command line as the –Argument parameter.

To query the diagnostics of an individual component use the following syntax:

Get-ExchangeDiagnosticInfo -Process EdgeTransport -Component SmtpOut –Argument help

The above for example shows the state of the process and that verbose is a supported argument. Other components have other arguments. For example, try verbose for SmtpOut

Get-ExchangeDiagnosticInfo -Process EdgeTransport -Component SmtpOut –Argument verbose

This shows you which hub transport servers and smarthosts are operational and reachable, along with the Configuration data which shows how often Exchange will check them to see if they are back online again.

Some of the output from this cmdlet returns a large amount of data. One such example of this is the ResourceManager component which returns the historical data for backpressure events on the server. Backpressure is when the server runs low or out of resources and so throttles or rejects first anonymous connections to the server and if resource utilisation does not return then goes into blocking all connections to see if resource utilisation can improve, and if it does, allowing connections back on again.

Hub Transport Load Balancing

Posted on 6 CommentsPosted in 2010, active directory, exchange, exchange online, powershell

In Exchange 2010 (not SP1) and Exchange 2007 there was no memory of unavailable transport servers and so the round robin method of load balancing across the hubs in the target delivery site or smarthosts used by connectors sourced to your current server was just that – round robin.

Though if a server was unavailable the next server in the list was selected and connected to and the first server in the list was moved to the end of the list of servers to use. This resulted in an uneven distribution of load when servers were offline. Imagine the scenario where you have three hub transports in the London Active Directory site (HL1, HL2 and HL3) which were installed in that order. A Hub Transport server in another AD Site will deliver up to 20 messages per connection and will make the connections in a round robin fashion. Therefore if HL1 is offline the connection will automatically be made to HL2. Upon completing the connection the first server in the list will be moved to the end of the list – in this example HL1 will move to the back of the list.

The next connection to the London site will use the list HL2, HL3, HL1 for delivery, and as HL2 is running will connect to HL2 and deliver its email and move HL2 to the back of the list. The third connect will go to HL3. The fourth connection will attempt to reach HL1 and fail, so deliver to HL2 and move HL1 to the back of the list.

The result of this is that HL2 will get 66% of email delivered to HL3’s 33% and not a 50/50 distribution once one server is down. When all servers in the site are operational the distribution will be 1/3 of connections each and even load balancing.

Exchange 2010 SP1 records downed servers in a separate list which it will attempt to connect to on a separate sequence (unrelated to email delivery). So taking the above example and HL1 is offline (again) and the source server is Exchange 2010 SP1 it will fail to connect and deliver to HL2, move HL2 to the bottom of the list and remove HL1 from the available servers list. Therefore HL2 and HL3 will get 50% of connections each – no overloading of the next hub in install order.

The source Exchange 2010 SP1 server will maintain this list of unavailable servers and will attempt to connect to the unavailable server regularly. It does this once a minute for four minutes (known as the QueueGlitchRetryCount and  QueueGlitchRetryInterval), then it changes to TransientFailureRetryCount and TransientFailureRetryInterval, which is six times, once every five minutes. After 35 minutes going through the Glitch and Transient retry intervals Exchange will only attempt to connect once every 10 minutes (the OutboundConnectionFailureRetryInterval value) or 15 minutes if on an Edge Transport server.

Once the server is online again it is added back into the round-robin load-balancing list for connections to remote sites or smarthost endpoints. This does mean though that if a server is offline for more than 35 minutes it will be up to 10 minutes before Exchange 2010 SP1 attempts to connect to it for transport and email delivery.

To see which servers are on your unavailable list run Get-ExchangeDiagnosticInfo -Process EdgeTransport -Component SmtpOut -Argument verbose . The Get-ExchangeDiagnosticInfo cmdlet is covered further in my next blog today.

Publishing ADFS Through ISA or TMG Server

Posted on 2 CommentsPosted in 2010, 2013, 64 bit, active directory, ADFS, ADFS 2.0, certificates, exchange, exchange online, https, isa, mcm, microsoft, Office 365, pki, tmg

To enable single sign-on in Office 365 and a variety of other applications you need to provide a federated authentication system. Microsoft’s free server software for this is currently Active Directory Federation Server 2.0 (ADFS), which is downloaded from Microsoft’s website.

ADFS is installed on a server within your organisation, and a trust (utilising trusted digital certificates) is set up with your partners. If you want to authenticate to the partner system from within your environment it is usual that your application connects to your AFDS server (as part of a bigger process that is better described here: http://blogs.msdn.com/b/plankytronixx/archive/2010/11/05/primer-federated-identity-in-a-nutshell.aspx). But if you are outside of your organisation, or the connection to ADFS is made by the partner rather than the application (and in Office 365 both of these take place) then you either need to install ADFS Proxy or publish the ADFS server through a firewall.

This subject of the blog is how to do this via ISA Server or TMG Server. In addition to configuring a standard HTTPS publishing rule you need to disable Link Translation and high-bit filtering on the HTTP filter to get it to work.

Here are the full steps to set up AFDS inside your organisation and have it published via ISA Server – TMG Server is to all intents and purposes the same, the UI just looks slightly different:

  1. New Web Site Publishing Rule. Provide a name.
  2. Select the Action (allow).
  3. Choose either a single website or load balancer or use ISA’s load balancing feature depending on the number of ADFS servers in your farm.
  4. Use SSL to connect:
    image
  5. Enter the Internal site name (which must be on the SSL certificate on the ADFS server and must be the same as the externally published name as well). Also enter the IP address of the server or configure the HOSTS files on the firewall to resolve this name as you do not want to loop back to the externally resolved name:
    image
  6. Enter /adfs/* as the path.
  7. Enter the ADFS published endpoint as the Public name (which will be subject or SAN on the certificate and will be the same certificate on the ADFS server and the ISA Server):
    image
  8. Select or create a suitable web listener. The properties of this will include listening on the external IP address that your ADFS namespace resolves to, over SSL only, using the certificate on your ADFS server (exported with private key and installed on ISA Server), no authentication.
  9. Allow the client to authenticate directly with the endpoint server:
    image
  10. All Users and then click Finish.
  11. Before you save your changes though, you need to make the following two changes
  12. Right-click the rule and select Configure HTTP:
    image
  13. Uncheck Block high bit characters and click OK.
  14. Double-click the rule to bring up its properties and change to the Link Translation tab. Uncheck Apply link translation to this rule:
    image
  15. Click OK and save your changes.

ADFS should now work through ISA or TMG assuming you have configured ADFS and your partner organisations correctly!

To test your ADFS service connect to your ADFS published endpoint from outside of TMG and visit https://fqdn-for-adfs/adfs/ls/idpinitiatedsignon.aspx to get a login screen

ERROR_REPLICA_SYNC_FAILED_THE TARGET PRINCIPAL NAME IS INCORRECT

Posted on Leave a commentPosted in 2003, 2007, active directory, error, exchange, kerberos, virtual pc, virtual server

This rather imposing message is found if you try to force replication between to Active Directory Domain Controllers when one of the controllers machine account password is out of sync with the password as stored on the other domain controller.

I have seen this a number of times on Virtual PC or Virtual Server Active Directory deployments with more than one DC in the virtual environment.

So, how do you fix it:

  1. On the DC that is broken (the one that when using replmon reports the error above) set the Kerberos Key Distribution Center Service to manual and stop the service.
  2. From a command prompt on the broken DC enter the following:
    netdom resetpwd /s:name_of_working_DC /ud:domain\user /pd:*
    where domain\user is an administrator of the domain in the domain_name\user_name format. You will be prompted to enter your password.
  3. Upon pressing Enter, if the command fails then restart the broken DC and repeat the above command (this restart clears the Kerberos ticket cache and so clears the broken credential attempts that it has stored).
  4. Upon successful completion of the command in step 2 restart the broken DC. You must do this even if done already in step 3.
  5. Check that replication is working, and if so restart the Kerberos Key Distribution Center Service and set the service back to automatic.

This is a summary of Microsoft Knowledgebase Article 325850, with some more specific detail mentioned.