Cannot Send Emails To Office 365 or Exchange Online Protection Using TLS

Posted on 7 CommentsPosted in 2003, 2007, 2010, 2013, exchange, exchange online, Exchange Online Protection, FOPE, hybrid, Office 365, spam, starttls, TLS

I have found this is a common issue. You set up an Exchange Online Hybrid or Exchange Online Protection (EOP) stand alone service and follow all the instructions for the creating of the connectors needed, only to find that your emails queue in your Exchange Server. If you turn on protocol logging you get this error in the log “Connector is configured to send mail only over TLS connections and remote doesn’t support TLS” and if you look at the SMTP protocol verbs that are recorded in the log you see that Microsoft’s servers do not offer STARTTLS as a verb.

STARTTLS is the SMTP verb needed to begin a secure and encrypted session using TLS. Communication between your on-premises servers and Microsoft for hybrid or EOP configurations requires TLS and if you cannot start TLS then your email will queue.

If you are not configuring hybrid or EOP standalone and need to send an email to someone on Office 365 then this is not an issue, because Exchange Server does not require TLS for normal email communication and so the lack of a STARTTLS verb means your email is sent in clear text.

The reason why you are not getting STARTTLS offered is that your connecting IP address is on the Microsoft block list. If you change your connector (temporarily) to allow opportunistic TLS or no TLS at all then your emails will leave the queue – but will be rejected by the Microsoft servers. The NDR for the rejection will tell you to email Microsoft’s delisting service. So now you have an NDR with the answer to the problem in, you can fix it! It takes 1 to 2 hours to get delisted from when Microsoft process your email – so they say it takes 48 hours end to end.

Therefore my recommendation when setting up Exchange Online Hybrid or stand alone EOP is to send an email over plain text to EOP before you configure your service. If you are on the blocklist then you will get back the delisting email and you can process that whilst setting up the connectors to Office 365 and so by the time you are ready to test, you are off the blocklist!

To send a test email over Telnet

  1. Install the Telnet Client feature on your Exchange Server that will be your source server for hybrid or connectivity to EOP for outbound email
  2. Type the following. This will send an email to a fake address at Microsoft, but will hit the TLS error before the message is rejected

    telnet microsoft-com.mail.protection.outlook.com 25

  3. You are now connected to Exchange Online Protection and you should get a 220 response
  4. Type the following to send the email by command line. No typo’s allowed in telnet, so type carefully. Replacing your email address where prompted so that you get the NDR back to you.

    ehlo yourdomainname.com
    mail from: youremailaddress@domainname.com
    rcpt to: madeupaddress@microsoft.com
    data
    from: Your Name <youremailaddress@domainname.com>
    to: madeupaddress@microsoft.com
    subject: testing to see if my IP is blocked

    type something here, it does not matter what, this is the body of the message you are sending
    .

  5. A few points about the above. It must finish with a . (full stop) on a line by itself followed by a carriage return. There must be a blank line between the subject line and the body. And finally, for each line of data you type, the Microsoft SMTP servers will return either a 250 or 354 response.

Access Is Denied Message After Sysprep–How To Fix

Posted on 1 CommentPosted in 2003, 2007, 2008, 2008 R2, 2012, 64 bit, backup, bios, hyper-v, password, recovery, sysprep, windows, windows 2003, windows 2008, windows 7, windows server, workstation, x64, x86

If before you use Sysprep to prepare a Windows machine for imaging you set the administrators password “User cannot change password” then sysprep will not clear this setting, but will set the “User must change password at next logon” setting. Normally these two settings are mutually exclusive, but in the scenario for sysprep it seems they can both end up being set.

This means you get prompted to reset you password at first logon after sysprep completes and then find you have “Access Denied” as the response. There is seemingly no way around this Catch-22.

That is unless you use the Offline NT Password and Registry Editor. This tool allows password resets when booting the server from a CD or USB key (so physical access to the server is required). As the download for this is an iso file, it can also be used in virtual environments by configuring your virtual machine to boot from the iso you have downloaded.

To allow you to logon to your machine following the above issue, all you need to in the Offline NT Password tool is to blank out the administrators password and unlock the account. These are options 1 and 4 during the password reset stage. Full instructions with screenshots follow:

  1. Boot the server with the issue with the Offline NT Password and Registry Editor iso file:
    image
  2. Choose the correct boot option (or just press Enter for the defaults):
    image
  3. For Vista and earlier select the default of Option 1. For Windows 7 and Windows 2008 and later select Option 2 (to boot into the second partition on the disk). You might need to select a different option if you have more partitions. You need to select the partition that Windows is installed on.
  4. If the disk is marked as Read-Only ensure that the server went through a clean boot and was not shutdown incorrectly. Once the messages indicate a writable partition
    image
  5. Select the presented folder (by pressing Enter again). You can typically just press Enter through most of these stages. You will be asked what you want to do – we want to reset passwords:
    image
  6. Select Option 1 to Edit user data and passwords:
    image
  7. Press Enter to choose the Administrator account:
    image
  8. Type 1 to Clear (blank) user password. You should get back the message “Password cleared!”:
    image
  9. Press Enter again to reselect the Administrator account, and this time select Option 4 to unlock the account (even though this program tells you the account is already unlocked):
    image
  10. Once you see “Unlocked!” you can quit from this program. The process to quit requires you to save your changes. Note that the default setting is not to save changes, so you cannot now use Enter to select the default option.
  11. Enter ! to quit from the password reset program:
    image
  12. Enter q to quit from the script and to ask about saving changes:
    image
  13. Enter y to write back the files that have been changed:
    image
  14. You should have been told “***** EDIT COMPLETE *****”. Press Enter to finish the program scripts:
    image
  15. At this final screen you can remove the CD or unmount the iso image from your virtual machine and press CTRL+ALT+DEL to restart the server. The server should now boot into Windows and auto-logon as it has a blank password.
  16. Change the password and optionally untick the “User cannot change password” setting.

Recovering Exchange Items When Entourage Corruption Deleted Them From The Server

Posted on Leave a commentPosted in 2003, Entourage, exchange, PFDAVAdmin, recovery

I have a client who has an Exchange 2003 server and uses Entourage for the Mac. Last week all his sent items disappeared. It seems the problem is that Entourage’s local database corrupted and sync’ed back to the server that all these items where now deleted!

To recover them I used PFDAVAdmin from http://www.microsoft.com/en-us/download/details.aspx?id=22427 and expanded the installation on the server to an empty folder. Within this empty folder I located pfdavadmin.exe and run the software:

image

Once the program is running, and ensure that you are logged in to the server as a user with a mailbox (as that seemed to stop the program working), click File and Connect:

image

Select All Mailboxes (I could not get the https:// URL for specified mailboxes to work, so I just went for All Mailboxes

Once the Getting Mailbox List dialog disappears you are left with PFDAVAdmin as before, just showing +Mailboxes. Expand Mailboxes, expand your selected problem mailbox, expand Top of Information Store and then locate the folder that contained the missing items. In my clients case it was Sent Items.

image

From the right hand screen select the Items tab. Wait for the items to appear and then from the bottom pick Deleted Contents (I’ve found if you change to Deleted Contents while the “Normal Contents” list is still being collected that PFDAVAdmin fails and you need to open a new copy of the program). The screen will update (eventually if you have lots of items) with a list of items that have Unknown for the Item-level Perms column. Select as many as you need to (I found PFDAVAdmin crashes if you pick more than 1000 items) and right-click the selection  to choose Recover Items

imageimage

Once recovery starts you need to wait a while based on the number of items recovered and then you will get dialog box popup confirming your selection. Press OK, or if the dialog is too pick to fit on the screen just press Enter, and recovery completes.

Repeat until you have all the items recovered, and I would recommend rebuilding the Entourage identity as well, as that database on Entourage is now to be considered suspect.

Highly Available Geo Redundancy with Outbound Send Connectors in Exchange 2003 and Later

Posted on 6 CommentsPosted in 2003, 2007, 2010, cloud, DNS, domain, door, exchange, exchange online, load balancer, loadbalancer, mcm, microsoft, MX, Office 365, smarthost

This is something I’ve been meaning to write down for a while. I wrote an answer for this question to LinkedIn about a week ago and I’ve just emailed a MCM Exchange consultant with this – so here we go…

If you configure a Send Connector (Exchange 2007 and 2010) or Exchange 2003 SMTP Connector with multiple smarthosts for delivery to, then Exchange will round-robin across them all equally. This gives high availability, as if a smarthost is unavailable then Exchange will pick the next one and mail will get delivered, but it does not give redundancy across sites. If you add a smarthost in a remote site to the send connector Exchange will use it in turn equally.

So how can get get geographical redundancy with outbound smarthosts? Quite easily it appears, and it all uses a feature of Exchange that’s been around for a while. But first these important points:

  • This works for smarthost delivery and not MX (i.e. DNS) delivery.
  • This is only useful for companies with multiple sites, internet connections in these sites and smarthosts in those sites.
  • This is typically done on your internet send connectors, the ones using the * address space.

You do this by creating a fake domain in DNS. Lets say smarthost.local and then creating A records in this zone for each SMTP smarthost (i.e. mail.oxford.smarthost.local). Then create an MX record for your first site (oxford.smarthost.local MX 10 mail.oxford.smarthost.local). Repeat for each site, where oxford is the site name of the first site in this example.

Then you create second MX records, lower priority, in any site but use the A record of a smarthost in a different site (oxford.smarthost.local MX 20 mail.cambridge.smarthost.local).

Then add oxford.smarthost.local as the target smarthost in the send connector. Exchange will look up the address in DNS as MX first, A record second, IP address last), so it will find the MX record and resolve the A records for the highest priority for the domain and then round-robin across these A records.

If you have more than one smarthost in a site, add more than one MX 10 record, one per smarthost. Exchange will round-robin across the 10’s. When all the 10’s are offline then Exchange will automatically route to mail.cambridge.smarthost.local (MX priority 20 for the oxford site) without needing to disable the connector and retry the queues.

If you used servernames and not MX’s then it would round-robin amongst all entries, and so equally sent email to Cambridge for delivery. The MX option keeps mail in site for delivery until it cannot and then sends it automatically to the failover site.

Unknown Error, Outlook 2003 and Exchange 2010

Posted on Leave a commentPosted in 2003, 2010, exchange, Outlook

It’s a well documented issue with Outlook 2003 connecting to Exchange 2010 that means Outlook 2003 is not as responsive in Online mode as it was with legacy versions of Exchange Server (http://support.microsoft.com/kb/2009942).

What is less well documented is an odd error message that can appear because of this interaction.

Imagine the following scenario. User on Outlook 2003 has lots of messages to delete, and deletes them one at a time. Outlook will not refresh the display for up to 5 seconds (the lowest setting that you can tell Outlook to refresh, via the Maximum Polling Frequency registry key). The problem is that if the user deletes a message and it does not disappear from the screen and then (thinking its gone, and the highlight has moved onto the next message) presses delete again. Outlook generates “Unknown Error” – which is not exactly helpful, and could appear as often as every other message that is deleted.

How to fix: Cached mode (though in the scenario I came across the above it was Outlook on a Terminal Server, so that’s not an option), upgrade the client version of Outlook, or use Shift or CTRL select and delete all your emails in one go!

Message Tracking Logs ‘And/Or’ Error

Posted on Leave a commentPosted in 2003, exchange

The following message appears in the Exchange Server 2003 message tracking logs:

The object ‘and/or’ in the message tracking logs can’t be found in the directory.  The object may have been deleted.  The tracking history may be incorrect.

This is an error that can be ignored as it does not mean that the email message that you are tracking has not been delivered (discounting other errors that is).

This error occurs in the Exchange Server 2003 tracking logs because the logs store the name of the recipients server, and if the recipients server is determined to be ‘and/or’ then that is what the log will read.

I have noticed that this error occurs when Exchange Server 2003 talks to Exim 4.69 servers with the following welcome message:

220-server.fqdn ESMTP Exim 4.69 #1 date time
220-We do not authorize the use of this system to transport unsolicited,
220 and/or bulk e-mail.

This welcome message is spread across three lines and the last line reads “220 and/or bulk e-mail”. Exchange uses the first word following 220 (and not 220) as the server name.

For example, the following is the welcome banner displayed by Postini/Google, which is a single line:

220 Postini ESMTP 108 y6_19_2c0 ready.  CA Business and Professions Code Section 17538.45 forbids use of this system for unsolicited electronic mail advertisements.

Therefore Exchange will identify this server in the message tracking logs as “Postini”.

Enabling ActiveSync on a Sony P1i with a GoDaddy Certificate

Posted on Leave a commentPosted in 2003, 2007, activesync, exchange, mobile phones, pki

GoDaddy issued certificates are not trusted by the Sony P1i phone and so if you are using a GoDaddy issued digital certificate for ActiveSync on one of these phones you will be prompted to accept the certificate at each sync. As this kills the purpose of push email sync you will want to stop the prompt.

You do this by installing the GoDaddy trusted root certificate. On any Windows computer that works when connecting to a website protected with your GoDaddy certificate run mmc.exe and add the Certificates snap-in, selecting the local user option. Browse to Trusted Root Certification Authorities and click on the Certificates node. Find and right-click the Go Daddy Class 2 Certification Authority and choose All Tasks > Export. Export the certificate as a DER encoded binary X.509 (.CER) file to a folder on that computer.

Email that file to the owner of the Sony P1i and sync the phone to download their email (confirming the prompt that we want to remove). Open the email and download the attachment. Once the attachment is downloaded (which might involve syncing again and confirming the certificate prompt) open the attachment. The phone will install the certificate into its certificate store. No more prompts!

ERROR_REPLICA_SYNC_FAILED_THE TARGET PRINCIPAL NAME IS INCORRECT

Posted on Leave a commentPosted in 2003, 2007, active directory, error, exchange, kerberos, virtual pc, virtual server

This rather imposing message is found if you try to force replication between to Active Directory Domain Controllers when one of the controllers machine account password is out of sync with the password as stored on the other domain controller.

I have seen this a number of times on Virtual PC or Virtual Server Active Directory deployments with more than one DC in the virtual environment.

So, how do you fix it:

  1. On the DC that is broken (the one that when using replmon reports the error above) set the Kerberos Key Distribution Center Service to manual and stop the service.
  2. From a command prompt on the broken DC enter the following:
    netdom resetpwd /s:name_of_working_DC /ud:domain\user /pd:*
    where domain\user is an administrator of the domain in the domain_name\user_name format. You will be prompted to enter your password.
  3. Upon pressing Enter, if the command fails then restart the broken DC and repeat the above command (this restart clears the Kerberos ticket cache and so clears the broken credential attempts that it has stored).
  4. Upon successful completion of the command in step 2 restart the broken DC. You must do this even if done already in step 3.
  5. Check that replication is working, and if so restart the Kerberos Key Distribution Center Service and set the service back to automatic.

This is a summary of Microsoft Knowledgebase Article 325850, with some more specific detail mentioned.

Dell Notebook System Software (NSS) Failure to Install

Posted on 1 CommentPosted in 2003, xp

I recently had to rebuild a Dell Latitude D610 as it was not working properly on purchase. The reinstall instructions for Windows XP SP2 included installing the relevant Dell drivers from the Dell CD (or internet download if they were later versions). The Notebook System Software (NSS) always failed to install – it would crash upon starting the program.

After phoning Dell Technical Support to fix this problem, the answer was to run the software whilst in Windows XP Safe Mode. To get to this reboot your computer and press F8 just before the Windows logo appears (easiest thing to do is press F8 repeatedly as soon as the computer boots and you are sure to get the option for Safe Mode).

After installing it in Safe Mode you can reboot into normal Windows XP.

Connecting a Windows SmartPhone to Exchange Server Protected with a Private Certification Authority Digital Certificate

Posted on 1 CommentPosted in 2003, certificates, exchange, orange, spv

Having recently obtained my first Windows Mobile powered SmartPhone, I needed to connect to my Exchange Server over the internet using ActiveSync. For those of you unfamiliar with Windows Mobile SmartPhones, they let you connect, using the phones internet connection (typically over a GPRS network), to your Exchange 2003 Servers to download your email at a given schedule. Additionally the SmartPhones running Windows Mobile 2003 and later support “Up-to-date Notifications”, where the emails are synchronised to your phone automatically upon arrival at the Exchange Server independent of the schedule. It was this Up-to-date Notifications feature that I wanted to implement, but it was not as straight forward as I thought it would be when I got down to it!

The reason was the phone. I have an Orange SPV C550 which is locked by Orange, the network operator. This means that you cannot install any software on the phone including any digital certificate that you need to connect to your Exchange Server.

To configure across the mobile network synchronisation of your e-mail you need to have Exchange ActiveSync enabled on your Exchange Server (it is on by default) and ensure that the “/Microsoft-Server-ActiveSync/*” path to an Exchange Server in your organisation is available through your firewall. If you do not use SSL to protect this HTTP session (not recommended) then you need do nothing to your phone apart from configure it to use the server synchronisation to get your email, but if you want to use HTTPS and the certification authority you are using to provide your digital certificates is a private certification authority you will find that you will not be able to connect as your phone will not trust the certificate issuer. Note that in test environments you can use the Disable Certificate Verification tool (see links below) to avoid this issue, but for a production network this is not recommended.

Therefore you need to unlock the phone and install the root certificate from your private certification authority and then relock the phone before you can make a secure connection to your Exchange Server from your Windows Mobile SmartPhone. The last step of locking your phone again is optional, but recommended as it maintains the security of your phone.

To unlock your Orange phone you need to follow these steps, though note that other mobile network operators will either provide unlocked phones or might have an equivalent process:

  1. Make at least one GPRS connection so that your device is registered at Orange
  2. That your handset is switched on and it has a good signal
  3. That you have a record of your IMEI number. You can get this by typing *#06# on the phone.
  4. Visit http://developer.orangews.com/orgspv/comdefq.aspx on a computer (you can do this on the phone, its just easier on a computer). At the time of writing this web page does not list the C550 phone as a phone it unlocks, but it does work.
  5. Choose to “Disable Certificate Security” and click Proceed. Enter the required information and your phone will make an internet connection (which you will be billed for) and it will unlock your phone. Once the phone is unlocked you will see a message in English and French telling you that “Your handset has had its certificate security disabled.”

Once the handset is unlocked you can install any application on the phone that you like, but for the purposes of connecting to your Exchange Server for Up-to-date Notifications:

  1. Start Internet Explorer on your phone and browse to a web site containing your root digital certificate (or use SPAddCert.exe if you already have the certificate downloaded to the phone’s memory. SPAddCert’s download location is on the list of links below). For example if your certificate server is the version that comes with Windows then visit http://servername/certsrv/certcarc.asp and download the certificate.
  2. Confirm that you want to install the certificate at the prompt. Assuming that the phone unlock was successful, the certificate will be installed.
  3. You can now relock your phone using the same process as described above, just choosing the “Enable Certificate Security” option instead. Though whilst your phone is unlocked you might want to investigate Global Contact Access from Microsoft (see the links below) to give your phone more access to your Exchange Server, such as the Global Address List and Free/Busy information.

Configuring Exchange ActiveSync on the Exchange Server is beyond the scope of this article, but full instructions can be found in the Microsoft Press Exchange Server 2003 Resource Kit on pages 892 onward to the end of the chapter.

Once you have the certificate installed you can configure the device to connect to the Exchange Server. This is done by starting the ActiveSync application on your phone and setting the options. Option 3, Server Settings controls this functionality and you need to choose menu item 4 (Connection). Here you need to enter your username, password and domain along with the server name, which is the web address to the Exchange ActiveSync server (for example mail.company.com). You can leave the SSL option selected as you now have the ability to do this connection securely, without needing to purchase a digital certificate from a public certification authority.

Links