The Total Data Received From the Remote Client Exceeded Allowed Maximum

Working with a large number of mailboxes is usually about the same a working with a small number of mailboxes, except that you need to include -ResultSize Unlimited in your Get-* cmdlets. However, I recently ran into the following error when getting a large list of mailboxes (approx 38K) with Get-Mailbox and piping them to Set-Mailbox:

Sending data to a remote command failed with the following error message: The total data received from the remote client exceeded allowed maximum. Allowed maximum is 524288000.

In the end this seemed to be a limitation of piping from Get-Mailbox to Set-Mailbox. For example, this would generate the error:

Get-Mailbox -ResultSize Unlimited | Set-Mailbox -SingleItemRecoveryEnabled $true

One option to work around this is to chunk up the work and do it by database instead:

Get-Mailbox -Database XXX -ResultSize Unlimited | Set-Mailbox -SingleItemRecoveryEnabled $true

However, my preferred workaround was this:

$mbx=Get-Mailbox -ResultSize Unlimited
foreach($m in $mbx) {Set-Mailbox $m -SingleItemRecoveryEnabled $true}

Using a foreach loop instead of piping seems to avoid the error when dealing with large data sets. The powershell.exe process for this task consumed about 2.5GB of memory.

List Disabled User Accounts with Mailboxes

I'm currently working with a large organization where they've lost track of mailbox removals when accounts are disabled. They keep disabled accounts for an extended period of time after users leave. So, some disabled accounts are years old and still have a mailbox.

We're going through and cleaning up right now. So, we wanted a list of disabled user accounts with an Exchange Mailbox. However, since resource mailboxes have a disabled user account, we need to exclude those. It's a nice easy one line of PowerShell:

Get-Mailbox -Filter {ExchangeUserAccountControl -eq "AccountDisabled" -and IsResource -eq $false} -ResultSize unlimited | export-csv C:\temp\DisabledWMailbox.csv

View Service Status for MTS

If you are having data connectivity issues on the MTS network this web site will show you the status of their services. Useful to figure out whether it's systemic or just you (or your customer).

• http://www.mts.ca/mts/support/service+status

Exchange 2010 Addressbook Errors with NetScaler 11.0

I'm working with a client that is upgrading the load balancing infrastructure that services their Exchange 2010 organization. They are implementing NetScaler SDX appliances that run NetScaler VPX virtual load balancers. Being as this is new infrastructure, version 11.0 (first release June 30, 2015, latest release Oct 8, 2015) is being implemented.

To test the new load balancing, we modified the hosts file on few computers to direct the load balanced names (outlook.domain.com/webmail.domain.com) to the IP address of the new load balancer. Initially, this seemed to be working fine, but we didn't do any specific testing.

After a bit, we noticed that we often got the following error in Outlook when accessing the address book:

The connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action.

This error was reproducible after not accessing the address book for a few minutes. In addition, when looking at the connection status, we would see failures talking to the directory. Connectivity to mail generally seemed fine, but also had occasional failures.

Connection Status with directory access failures

When you research this problem, you will find lots of references to incorrect network timeouts closing connections before the application is ready. This is exactly what we seemed to be having. However, we adjusted every possible setting on the NetScaler to allow connections to last up to two hours. No matter what we adjusted, the same problem remained.

Other network components were the same as before. So, it seemed the problem had to be the new NetScaler. Clients still using the existing load balancer did not have any issues.

After adjusting everything we could think of, we called Citrix support. In conjunction with support, the network team did some packet traces and confirmed that resets were being performed on the RPC connections for the address book.

The fix ended up being using version 10.5 (latest release Sept 2015) instead of version 11.0 for the VPX instance. As soon as version 10.5 was implemented the mysterious reset issue was resolved and the response time for the directory lookups dropped significantly.

Connection Status after implementing version 10.5

Citrix is continuing to evaluate the issue and I'll update when I have more information.

Update (Nov 10/15): Apparently NetScaler 11.0 has issues with Outlook Anywhere also: http://discussions.citrix.com/topic/368750-netscaler-11-6210-outlook-anywhere-broken/page-2

Update (Nov 13/15): Running version 10.5 we've still had no issues. It has been submitted as a bug with Citrix and we're waiting for a update. Hopefully the next revision will fix it, but they are definitely aware it's an issue.

Update (Nov 27/15): Citrix did some testing with the network folks again today with Version 11.0 (I'm assuming a new build) and still not working properly. We are back to Version 10.5.

Resources:

• NetScaler Release Software Dates: http://support.citrix.com/article/CTX121840
• http://blogs.technet.com/b/david231/archive/2015/03/30/for-exchange-2010-and-2013-do-this-before-calling-microsoft.aspx
• https://discussions.citrix.com/topic/348704-exchange-2010-rpc-timeout-issue/
• https://social.technet.microsoft.com/Forums/exchange/en-US/e7c5ab7c-7ccc-4f4a-ac40-31831aea2d81/outlook-time-out-setting?forum=exchange2010
• https://masterxen.wordpress.com/2015/08/11/exchange-rpc-timeouts-when-using-citrix-netscaler/

Query Mailbox by LegacyExchangeDN

In a large organization with mailbox quotas, you will see an ongoing number of events indicating that users are at the limit of their quotas. In the event log you will see:

Event Source: MSExchangeIS
Event ID: 8528
The mailbox for /o=ExchangeOrg/ou=AdministrativeGroup/cn=Recipients/cn=UserIdentifier has exceeded the maximum mailbox size. This mailbox cannot send or receive messages. Incoming messages to this mailbox are returned to sender.  The mailbox owner should be notified about the condition of the mailbox as soon as possible.

The overall identifier for the mailbox is the LegacyExchangeDN attribute for the user mailbox. Most of the time, you can look at this value and quickly figure out which user is having the issue. If it's a current user, you can wait or contact them. In a large environment, you may not personally know the users, and you can investigate whether this is a disabled account that needs to be removed.

Today I was looking at one of these events and saw that the final UserIdentifier portion of the LegacyExchangeDN attribute was a long GUID number. To find the actual user account I needed to query the mailbox with a filter for LegacyExchangeDN.

Get-Mailbox -Filter {LegacyExchangeDN -eq " /o=ExchangeOrg/ou=AdministrativeGroup/cn=Recipients/cn=UserIdentifier"}

Please note that using the -Filter parameter is much more efficient that doing Get-Mailbox and piping it to Where-Object. Using the -Filter parameter is almost instant whereas piping Get-Mailbox to Where-Object can take several minutes.