Putting Office 365 Room Mailboxes in Local Exchange

Recently I was working with an organization that had both an Office 365 tenant and on-premises Exchange 2007. Our project was to merge these two together into a single unit by configuring hybrid mode.

As part of this process, there are local AD user accounts that needed to be linked to Office 365 mailboxes in a way that the local Exchange implementation could understand. I've described that process here:

• http://byronwright.blogspot.ca/2013/11/converting-mail-user-to-remote-user.html

The existing Office 365 tenant has some room mailboxes. In order to allow on premises users to book those room, we need to perform a similar process for the room mailbox.

Here is the process I used:

1. Create a disabled user account with the same name as the O365 room.
2. Convert the disabled user to a mail user:
1. Set the External e-mail address to be for the O365 object. This should be XX@XX.mail.onmicrosoft.com.
3. Set the local domain as the reply email address. This needs to match the address in O365 because that is how Dirsync matches the disabled user account to the O365 object.
1. On the E-Mail addresses tab, uncheck the Automatically update e-mail addresses based on e-mail address policy check box.
2. Select the correct e-mail address XX@yourdomain.com and click Set as Reply.
3. Click Apply.
4. Check the Automatically update e-mail addresses based on e-mail address policy check box and click Apply.
4. Use AD Users and Computers to Edit the properties of the disabled user account. Advanced Features must be enabled in the View menu.
1. On the Attribute Editor tab, modify the following values to convert the disabled mail user to a remote room mailbox:
• msExchRecipientDisplayType: -2147481850
• msExchRecipientTypeDetails: 85899334592
• msExchRemoteRecipientType: 33

Finally, run Dirsync to replicate the object to O365. The object should be matched with the existing o365 room mailbox. You can now book meetings with the room from your on-premises Exchange.

UPDATE:
The above process seemed to work well in my personal environment with an Exchange 2010 hybrid server. However, on a recent project with an Exchange 2013 hybrid server, it didn't seem to work at all.

What we did as an alternative was link the room mailbox in O365 with a disabled mail user. Then we setup the proper mailbox ID to allow mailbox moves. Then finally, we moved the room mailbox from O365 to on-premises and then back to O365. This gave a properly configured room mailbox in O365 that showed up properly in the Exchange 2013 management tools.

IP Addresses for Office 365

When implementing a hybrid configuration of Exchange Server and Office 365, the configuration wizard automatically limits connectivity to the Exchange server by setting address ranges on the receive connector of Office 365 in the on-premises Exchange. If you want the additional security of limiting access to the hybrid server at the firewall level, you can copy the configured addresses into your firewall rule.

Alternatively, if you prefer the following links provide lists of the IP addresses used by Office 365 services.

Exchange Online Protection (EOP):

•  http://technet.microsoft.com/en-us/library/dn163583%28v=exchg.150%29.aspx

Forefront Online Protection for Exchange (FOPE)

• http://technet.microsoft.com/en-us/library/hh510075.aspx

Various Office 365 services:

•  http://technet.microsoft.com/en-us/library/hh373144.aspx

Replace Missing Cluster Name Object for DAG

Ran into a strange issue today with an Exchange 2013 DAG. The cluster name object (CNO) for the DAG had been deleted at some point. It must have been a long time ago because I couldn't find a tombstone objects for it to try and bring it back. What was amazing is that the DAG functioned fine except that the File Share Witness (FSW) was offline because the CNO is used to access the shared folder for the FSW.

When you create a DAG, a computer object is created that represents the cluster name. This is the CNO.

A quick search revealed a number of documents talking about how to recover the DAG object from Deleted Items. However, this was not possible for me. Instead, I had to recreate the CNO.

Before we go any further, let me say that the smart thing to do is probably break the DAG and recreate it in this scenario. If you have up to date copies of the data in the remote location, then adding the replica back after you recreate DAG should go quickly. However, I figured I'd give the repair a try. Let's just say this is not exactly MS approved.

First I searched and searched to make sure that the CNO was really not there. The CNO has the same name as the DAG and mine was not there.

Here is the process I followed:

1. Create a new computer account for the CNO named the same as the DAG.
2. Give the DAG nodes (computer accounts) full control permission to the CNO.
3. Identify the objectGUID attribute for the CNO and make note of it. This will be added to the cluster configuration.
4. On one of the DAG nodes, use Regedit and view HKLM\Cluster and identify the value of ClusterNameResource. It will be a long ugly number and is the Cluster Name Resource GUID
5. In Regedit, browse to HKLM\Cluster\Resources\ClusterNameResourceGUID\Parameters.
6. Edit the ObjectGUID key and replace the value with the value you copied in step 3. This tells the cluster to use the new CNO.
7. Perform steps 5 and 6 on all DAG nodes.
8. Restart the Cluster service on all DAG nodes.
9. Update share permissions on the FSW shared folder to give the CNO full control.
10. Update ntfs permissions on the FSW folder to give the CNO modify.

NOTE: In step 3, you cannot use the GUID value as shown in the Attribute Editor tab in AD Users and Computers. You need to view the objectGUID attribute and copy the Hexadecimal value. This is a reordered version of the value visible in the Attribute Editor tab.

At this point, I was still getting errors about the File Share Witness not being able to start because of failed logon. The event log had the following error:

Event ID: 1228

Cluster network name resource 'Cluster Name' encountered an error enabling the network name on this node. The reason for the failure was: 'Unable to obtain a logon token'.
The error code was '1326'.

The final step I needed to perform was:

1. In Failover Cluster Manager, take the cluster name offline.
2. Right-click the cluster name, point to More Actions, and click Repair.

The repair seemed to sync up the computer account name and computer account password with the failover cluster. I'm not sure this is entirely accurate, but at this point the cluster name started with no errors in the event log and the FSW resource started properly.

Some of the errors I received during troubleshooting were:

Event ID: 1069
Cluster resource 'File Share Witness (\\fsw.domain.com\dag.domain.com)' of type 'File Share Witness' in clustered role 'Cluster Group' failed.
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.

Event ID: 1196
Cluster network name resource 'Cluster Name' failed registration of one or more associated DNS name(s) for the following reason: The handle is invalid.
Ensure that the network adapters associated with dependent IP address resources are configured with at least one accessible DNS server.

Event ID: 1207
The computer object associated with the cluster network name resource 'Cluster Name' could not be updated in domain 'domain.com' during the
Resource post online operation.
The text for the associated error code is: There is no such object on the server.
The cluster identity 'CNO$' may lack permissions required to update the object. Please work with your domain administrator to ensure that the cluster identity can update computer objects in the domain.

Some resources that were useful for troubleshooting:

• http://blogs.technet.com/b/askcore/archive/2009/04/27/recovering-a-deleted-cluster-name-object-cno-in-a-windows-server-2008-failover-cluster.aspx. A comment by Fred on this blog describes recovery of the CNO for Windows Server 2012, which is what I did.
• http://social.technet.microsoft.com/Forums/windowsserver/en-US/2ad0afaf-8d86-4f16-b748-49bf9ac447a3/ws2012-cluster-network-dns-issues?forum=winserverClustering. This blog clued me in to repairing the CNO.
• 
  

Microsoft Exchange Search Host Controller Failing

Got a call from a client that reported user searches in OWA and Outlook were failing. This particular client had recently has some severe SAN issues. So, who knows were the problem could be. However, the place to start is with the indexes, and the index on this server showed as healthy.

Next, I looked at the services. Microsoft Exchange Search was running properly, but Microsoft Exchange Search Host Controller was stopped. When I started Microsoft Exchange Search Host Controller, it showed as running for a few seconds and then stopped.

In the event log, two related events were showing:

Event ID: 1026
Source: .NET Runtime
Application: hostcontrollerservice.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: Microsoft.Ceres.HostController.Controller.HostControllerException
Stack:
   at Microsoft.Ceres.HostController.WcfServer.WcfService.StartService()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.ThreadHelper.ThreadStart()

Event ID: 1000
Source: Application Error Faulting application name: hostcontrollerservice.exe, version: 15.0.4454.1006, time stamp: 0x50d08ef5 Faulting module name: KERNELBASE.dll, version: 6.2.9200.16857, time stamp: 0x530e784b
Exception code: 0xe0434352
Fault offset: 0x0000000000047b8c
Faulting process id: 0x413c
Faulting application start time: 0x01cf86480e45a787
Faulting application path: C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\hostcontrollerservice.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 4cc1a310-f23b-11e3-943e-00155d09980c
Faulting package full name:
Faulting package-relative application ID:

I found a couple of references to a corrupt and empty ini file that can cause this issue. The file in question is C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data\Nodes\Fsis\node.ini. However, mine appeared to be good. There was content in the file in any case.

My fix ended up being removing and reinstalling the search services. This was based on advice given to another person by Microsoft support. The steps were:

1. Ensure that both Microsoft Exchange Search and Microsoft Exchange Search Host Controller are stopped. I had to manually kill noderunner.exe processes that were started by Microsoft Exchange Search Host Controller but not stopped when the service failed.
2. Create a backup copy of InstallConfig.ps1 in C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\Installer.
3. Edit InstallConfig.ps1 and modify the line for the baseport variable to be $script:baseport:3800. On this system the original value was 17000. I'm not sure if that's consistent or varies across systems.
4. Uninstall search by running: .\InstallConfig.ps1 -u -datafolder "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"
5. Reinstall search by running:   .\InstallConfig.ps1 -i -datafolder "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"

Other links for research into this issue:

• http://social.technet.microsoft.com/Forums/exchange/en-US/74aef23c-fe67-4f2f-a74e-4e895db28fa9/microsoft-exchange-search-host-controller-service-stopped-no-update-installed?forum=exchangesvrgeneral
• http://plasmasphere.co.uk/exchange-2013/exchange-2013-hostcontrollerservice-exe-crashes/

Estimator for Disk IOPS

One of the things I've always had issues with is finding a good way to estimate disk IOPS when I'm planning out systems. This is particularly important now with Hyper-V hosts and multiple VMs.

Today by luck, I saw someone in a forum talking about web page to provide estimated IOPS. I took a look and I love it.

This calculator lets you enter in the type of drive, hard drive size, number of disks, read percentage, and write percentage. Based this it gives you estimated IOPS for RAID 5,6,and 10.

The tools is here:

• http://www.thecloudcalculator.com/calculators/disk-raid-and-iops.html