Friday, November 28, 2014

Outlook 2013 - Constant restarts required

During a recent migration from Exchange 2007 to Exchange 2013, we ran into an issue that I was unaware of. Outlook 2013 was popping up multiple times per day with the following message:
The Microsoft Exchange administrator has mad a change that requires you to quit and restart Outlook.
After doing some research this appears when the default public folder database for a Mailbox database is set to a non-existent public folder database. In our scenario, we removed Exchange 2007 public folders after introducing Exchange 2013. Consequently, the Exchange 2013 databases were configured to use the Exchange 2007 public folder database as default. After removing the Exchange 2007 public folder database, the Exchange 2013 database were still configured to use the deleted public folder database that was now in deleted objects.

This issue seems to be specific to Outlook 2013 and Exchange 2013 CU5. I have not applied CU6 because there are known compatibility issues with Exchange 2007 coexistence for ActiveSync.

The fix for this was to use ADSI Edit and clear the msExchHomePublicMDB attribute on the database object. The databases for Exchange 2013 are located in: CN=Configure,DC=domain,DC=com > CN=Services > CN=Microsoft Exchange > CN=OrganizationName > CN=Administrative Groups > CN=Exchange Administrative Group (FYDIBOHF23SPDLT) > CN=Databases.

Properties of Mailbox Database in ADSI Edit

For more information about this issue, see:
UPDATE: We were also running into a strange issue with Outlook 2007 where clients would connect and be able to send messages immediately but not send for a few minutes. That issue was also resolved by removing the reference to the old public folder database.

Monday, November 24, 2014

Searching the Message Tracking Log for Virus Recipients

We had a client get infected with a virus today that used Outlook for sending messages out. This was unusual. Most viruses attempt to deliver email themselves and can be blocked by the firewall. Because this virus used Outlook, the messages were sent through the Exchange server which is allowed to deliver email to the Internet.

Three specific users got infected and I wanted to be able to inform those that were sent messages not to open them. I could get this information from the message tracking log based on the subject of the message. I used three commands to dump the information to a text file.

First, get the list of messages sent by a specific user:
$UserMessages=Get-MessageTrackingLog -Start "MM/DD/YYY 00:00:00AM" -Resultsize Unlimited | Where-Object {$_.MessageSubject -like "SubjectOfVirusMessage" -and $_.EventID -like "Send" -and $_.Sender -like "SenderEmailAddress"

Second, build a list of message recpients:
ForEach ($m in $UserMessages) {$UserRecipients=$UserRecipients+$_.Recipients}

Finally, dump to text file.
$UserRecipients > C:\UserRecipients.txt

I then provided the text file to each user to inform the necessary recipients.


Removing a Message from All Mailboxes

This morning a client got a virus infected email message that got sent to all users. Only 3 users opened the infected attachment initially, but I didn't want to count on users to delete the message manually. The risk of one forgetting and opening the attachment was too high.

In this case the virus was in an attachment named document2174_pdf.zip. This filename was unique enough that I was comfortable searching for all instances of that attachment and deleting the messages. If the filename was more generic, I would have included the message subject in the query.

Before you can perform this type of search, your user account must be a member of the Discovery Management group. This is required to do multi-mailbox search. Group membership does not take effect immediately, you may need to close and reopen your Exchange Management Shell prompt.

The syntax I used to delete all messages with the specific attachment from all mailboxes was:
Get-Mailbox -Resultsize Unlimited | Search-Mailbox -SearchQuery "attachment:document2174_pdf.zip" -DeleteContent
Note: This deletes the entire message, not just the attachment.

Friday, November 21, 2014

Missing Email Attachment

A few weeks ago, a client reported that when sending to a list of email addresses, one of the recipients was not getting the attachment. The recipient got the email message, but not the attachment. So, then you start to work it through logically:
  • Others get the attachment, so, it must not be on our end.
  • Send a test message from a non-client account and the attachment goes through, so, it must be on our end.
  • Apparently it's nobody's fault.
Of course it's always someone's fault, and it this case, I'm blaming Outlook. This turned out to be a new problem I've never seen with rich text formatting (RTF).

I'm sure that everyone that supports Outlook has at one time or another run into the winmail.dat attachment problem when Outlook sends RTF formatted email to a client that doesn't understand it. In this case, the client receiving the email partially understood RTF. It understood it enough to get the text message out, but not the attachment. The whole message was being delivered, but the client software dropped the attachment.

Outlook allows you to define on a per contact basis what formatting will be used when sending a message to a recipient. I'm yet to find a user that sets this on purpose, but in this case, it was accidentally set for RTF.

Rather than trying to fix this at the Outlook level and to ensure that it never happens again, I've disable RTF at the Exchange server level. If the client sends an RTF formatted message, Exchange will change it to HTML which is much more widely supported.

In Exchange Server, you can set how RTF format is used per domain. By default, there is a single domain in Organization Configuration > Hub Transport > Remote Domains, named Default. On the Message Format tab for this domain, under Exchange rich-text format, select Never Use. After doing this attachments were delivered correctly.

Properties of Default Remote Domain

Monday, November 17, 2014

Disable Inbound Proxy Probe Messages for Journaling

During a recent Exchange 2007 to Exchange 2013 migration, we ran into an issue with Managed Availability in Exchange 2013. Managed Availability provides health monitoring services.

This particular client is performing journaling by enabling journaling on each database and configuring all messages to be sent to a journaling mailbox in a separate database. They then have archiving software that removes the messages from the journaling mailbox. This allows them to retain a copy of all messages sent or received in the organization.

As we implemented Exchange 2013, the journaling mailbox was filled with messages generated by Managed Availability. The messages were from inboundproxy@contoso.com with a subject of Inbound proxy probe to a health mailbox.

Managed Availability was sending messages through the transport system to health mailboxes to verify that the system was functioning normally. However, this was resulting in a lot of messages that were not required being journaled. To avoid this, there are two options:
  1. Use a Journaling rule for specific users or group membership instead.
  2. Disable some monitors in Managed Availability.
I'm always reluctant to disable basic functionality in the system. So, our first attempt was by using a journaling rule with a dynamic distribution group for all users. When we did this we found that it was not reliably journaling the messages. So, we abandoned the journaling rule option and chose to disable the monitors generating the email messages instead.

In a knowledgebase article, Microsoft provides the following three commands for disabling the messages:
Add-GlobalMonitoringOverride -Identity "FrontendTransport\OnPremisesSmtpClientSubmission" -PropertyName Enabled -PropertyValue 0 -ApplyVersion "15.0.620.29" -ItemType Probe

Add-GlobalMonitoringOverride -Identity "MailboxTransport\Mapi.Submit.Probe" -PropertyName Enabled -PropertyValue 0 -ApplyVersion "15.0.620.29" -ItemType Probe

Add-GlobalMonitoringOverride -Identity "FrontendTransport\OnPremisesInboundProxy" -PropertyName Enabled -PropertyValue 0 -ApplyVersion "15.0.620.29" -ItemType Probe

Note:
In these commands, you must provide the -ApplyVersion parameter which specifies which version of Exchange Server that the override applies to. When you apply new updates to your system, you must run these commands again.

To identify the version of Exchange you are running, you can use the following command:
Get-ExchangeServer | FL Name,AdminDisplayVersion

The Microsoft documentation does not talk about it, but unless you have a single database, you have more than one OnPremisesInboundProxy monitor. There is an OnPremisesInboundProxy monitor for each database and is unique for the health mailbox associated with each database and you need create an override for each one. On this system with four mailbox databases, the identity of each monitor was:
  • FrontendTransport\OnPremisesInboundProxy
  • FrontendTransport\OnPremisesInboundProxy_2
  • FrontendTransport\OnPremisesInboundProxy_3
  • FrontendTransport\OnPremisesInboundProxy_4
To identify all of the OnPremisesInboundProxy monitors on your system, you can use the following command:
(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/ProbeDefinition | % {[XML]$_.toXml()}).event.userData.eventXml | ?{$_.Name -like "OnPremisesInboundProxy*"}

After creating an override for each instance of OnPremisesInboundProxy, all of the Inbound Proxy Probe messages were disabled.

Note:
To ensure that the monitoring overrides take effect, you need to restart the Microsoft Exchange Diagnostics and Microsoft Exchange Health Manager services on all Exchange 2013 servers.

Additional resources:

iDRAC7 - No Mouse or Keyboard

We recently had an emergency issue with a client system and had issues with the iDRAC7 remote control features. We put iDRAC cards in all of our servers for exactly this scenario where there is an unknown issue and we might need to power cycle the system.

We were able to log into the iDRAC7 web console with no issues. However, on the summary screen it was not showing status for the power supplies. Also, if I tried to browse the disk status, it was indicating the no disks or controllers could be seen.

Fortunately, the fix was pretty easy. Just restart the iDRAC card. To do this, on the Summary screen, in the Quick Launch Tasks, click Reset iDRAC as shown below. This does not reset the configuration of the iDRAC, it just restarts the card.

Wednesday, November 5, 2014

Require Encryption for a Specific Email Domain

You might not realize it, but your Exchange server is probably already encrypting messages sent and received from the Internet. By default, Exchange uses opportunistic TLS. This means that it offers TLS for inbound messages, but does not require it. Exchange also tries to use TLS for outbound messages, but does not require it.

We have one client that works with an insurance company. In order to ensure that data is secure, they request that their customers force the use TLS instead of relying on opportunistic TLS. This is more secure because the messages will wait in the queue if TLS cannot be established.

To use TLS for inbound messages, you need to have a valid certificate installed on your Exchange server and have assigned the SMTP service to that certificate. That certificate needs to include the name that external servers use to reach your server, such as mail.conexion.ca. As long as Transport Layer Security is enabled as an authentication mechanism on the Receive connector, opportunistic TLS is used for inbound messages.


You do not need to do anything to use TLS for outbound messages. TLS for outbound message relies on the certificate of the recipient server. However, you can enforce the use of TLS for specific domains by creating a send connector for those domains. Then after the send connector is created, you can use the Exchange Management Shell (EMS) to for TLS for that send connector by using the following command:
Set-SendConnector TLSConnector -RequireTLS $True
You can also force TLS for a receive connector, however, those are based on IP address. If the sender changes the IP address, then TLS will not longer be required. So, in most cases opportunistic TLS is a better choice for inbound messages.

Note: If you have another proxying device like an antispam appliance between Exchange and the Internet then you need to setup encryption on that device rather than your Exchange server.