Wednesday, July 29, 2020

Issues with Exchange 2010 and Exchange 2016 coexistence

This one is primarily notes to myself...

  • When Exchange 2016 is installed with Exchange 2010, MAPI over HTTP is enabled by default for the organization.
  • Exchange 2010 mailboxes continue to accessed via RPC.
  • Exchange 2016 mailboxes will use MAPI over HTTP
  • If Exchange 2016 mailboxes have Full Access to an Exchange 2010 mailbox then Outlook Anywhere is used to connect to that secondary mailbox.
  • If using a wildcard cert you need to set the certificate name for the EXPR outlook provider for Outlook Anywhere as msstd:*domain.com.

Authentication prompts

Exchange 2010 on Windows Server 2008 R2 requires a security update for Outlook Anywhere to function properly. This is a security update from 2016 (KB3140410). It "should" already be in place, but if it's not then Outlook Anywhere will cause tons of authentication popups in Outlook.


I saw this manifest as Exchange 2016 mailboxes with a secondary mailbox on Exchange 2010 getting the popups. Only an Exchange 2016 mailbox was fine because it used only MAPI over HTTP on  Exchange 2016. Only an Exchange 2010 mailbox was fine because it used only RPC to Exchange 2010.

If the update is not in place and you don't have the opportunity to apply the update quickly, you can modify DefaultAppPool in IIS Manager to use the identity Network Service. Recycle DefaultAppPool for the the change to take effect. Recycling DefaultAppPool does not affect users.

Error message that you will likely see in RpcHTTP proxy log (C:\Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\RpcHttp):
Complete=PrepareServerRequest;,WebExceptionStatus=ProtocolError;ResponseStatusCode= 401;
WebException=System.Net.WebException: The remote server returned an error: (401) Unauthorized. at
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at
Microsoft.Exchange.HttpProxy.RpcHttpProxyRequestHandler.<>c__DisplayClass1.nullb__0 
();HttpException=System.Web.HttpException (0x80004005): NegotiateSecurityContext failed with for 
host 'mail.contoso.com' with status 'InvalidToken' at 
Microsoft.Exchange.HttpProxy.KerberosUtilities.GenerateKerberosAuthHeader.

Win7 certificate errors

Windows 7 clients that don't have TLS 1.1 and 1.2 enabled might see a certificate error when connecting to Exchange 2016 for web services (not necessarily mailbox). To enable TLS 1.1 and 1.2 on Windows 7, you need to ensure that update KB3140245 is installed. With the update installed, you need to create additional registry entries.
The registry keys created by the quickfix utility distributed with this update by Microsoft are:

  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\WinHttp
    • Create DWORD: DefaultSecureProtocols
    • Value: 0xA00
  • HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Internet Settings\WinHttp
    • Create DWORD: DefaultSecureProtocols
    • Value: 0xA00
  • HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings
    • Create DWORD: SecureProtocols
    • Value: 0xA8
  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings
    • Create DWORD: SecureProtocols
    • Value: 0xA8
The DefaultSecureProtocols key is used by the Office Apps and the value 0xA00 designates TLS 1.1 and TLS 1.2.

The SecureProtocols key is used by Internet Explorer and the value 0xA08 designates TLS 1.0, TLS 1.1, and TLS 1.2.

Tuesday, April 14, 2020

MIS2000 Links

Using Power BI for data analytics and reporting
https://docs.microsoft.com/en-us/learn/paths/create-use-analytics-reports-power-bi/

City of Winnipeg Software Piracy
https://www.cbc.ca/news/canada/manitoba/city-of-winnipeg-manager-in-charge-of-police-radios-arrested-after-2-year-investigation-1.5027975

What's wrong with this picture?
https://twitter.com/DaveLeeBBC/status/1102359402151985152

Computer system failure grounds transit system in San Francisco
https://www.bart.gov/news/articles/2019/news20190309

Supply Chain Management Simulator
https://www.scmglobe.com/supply-chains-roman-empire/

Career wisdom from IT pros
https://www.reddit.com/r/sysadmin/comments/dzm3xs/once_a_young_sysadmin_and_now_an_old_unicorn_how/

(A few) Ops Lessons We All Learn The Hard Way
https://www.netmeister.org/blog/ops-lessons.html

Michael Geist blog (copyright and net freedom issues)
http://www.michaelgeist.ca/


Government IT failures


Federal Government - Phoenix payroll system #1
http://www.oag-bvg.gc.ca/internet/English/parl_oag_201711_01_e_42666.html

Federal Government - Phoenix payroll system #2
http://www.oag-bvg.gc.ca/internet/English/parl_oag_201805_01_e_43033.html

More Phoenix payroll
https://twitter.com/PSSuzanne/status/1098720582516895745

Phoenix payroll - 3 years later and still broken
https://www.thestar.com/news/canada/2019/03/10/still-stuck-on-the-phoenix-pay-roller-coaster-these-canadians-just-want-to-get-off.html

Federal Government - Gun registry
https://www.cbc.ca/news/canada/timeline-the-gun-registry-debate-1.786548

Security

Malware distribution hosted in LED light control console
https://news.microsoft.com/apac/features/law-enforcement-and-microsoft-come-together-to-bust-a-major-malware-attack-in-taiwan/

Ransomware attacks lock 2 Manitoba law firms out of computer systems
https://www.cbc.ca/news/canada/manitoba/winnipeg-law-firms-computer-virus-ransomware-1.5530825

Is Huawei really a risk?
https://www.reuters.com/article/us-huawei-tech-usa-pompeo/u-s-wont-partner-with-countries-that-use-huawei-systems-pompeo-idUSKCN1QA1O6?utm_source=reddit.com

City of Saskatoon phishing
https://thestarphoenix.com/opinion/columnists/tank-scam-stings-at-saskatoon-city-hall-with-shiny-fiscal-reputation

City of Ottawa victim of phishing
https://www.ctvnews.ca/canada/ottawa-city-treasurer-transfers-130k-of-taxpayer-funds-to-email-fraudsters-1.4371900

Stolen laptop with health data
https://www.cbc.ca/news/canada/north/nwt-stolen-laptop-encryption-it-department-1.5044118

Government of Nunavut Ransomware
https://nunatsiaq.com/stories/article/government-of-nunavut-slowly-rebuilds-computer-network-following-ransomware-attack/

Nursing Home Network Ransomware
https://www.cbsnews.com/news/hackers-ransomware-nursing-homes-14-million/

Spear Phishing
https://www.microsoft.com/security/blog/2019/12/02/spear-phishing-campaigns-sharper-than-you-think/

Scammy companies


WeWork - A tech company?
https://www.businessinsider.com/weworks-nightmare-ipo?r=US&IR=T?utm_source=markets&utm_medium=ingest

WeWTF
https://www.profgalloway.com/wewtf

Theranos timeline
https://www.refinery29.com/en-ca/2019/01/223033/theranos-scandal-timeline-what-happened-elizabeth-holmes-documentary

Theranos - Wall Street Journal expose
https://www.wsj.com/articles/theranos-has-struggled-with-blood-tests-1444881901


Cryptocurrency


What is cryptocurrency?
https://blockgeeks.com/guides/what-is-cryptocurrency/

QuadrigaCX Cryptocurrency Exchange Shadiness
https://www.cbc.ca/news/business/quadriga-bitcoin-bankruptcy-1.5004735

QuadrigaCX empty accounts - even more shady
https://www.ctvnews.ca/business/search-into-missing-cryptocurrency-turns-up-empty-cold-wallets-report-1.4319270

Cryptoqueen: How this woman scammed the world, then vanished
https://www.bbc.com/news/stories-50435014




Thursday, April 2, 2020

Azure AD Connect Large Object Error

A client is migrating their remaining mailboxes from on-premises Exchange to Office 365. Today they went to migrate a mailbox, but the user account wasn't replicated up to Office 365. After verifying that it was not being filtered by OU in Azure AD Connect, I checked the Synchronization Service Manager for Azure AD Connect and found an error listed for the export to the Azure AD tenant (XXX.onmicrosoft.com).

The error was LargeObject and when I drilled down, it had these details:
The provisioned object is too large. Trim the number of attribute values on this object.

This error is typically caused by:
  • Too many user certificates (15 max)
  • Too many SMIME certificates (15 max)
  • A thumbnail photo that is too large
  • Too many proxy addresses
This user object did not have any user certificates, SMIME certificates, or a thumbnail photo. So, let's check out the proxy addresses.


The user object had 540 addresses. After a bit more research, I found that user objects in Azure AD have a limit of 400 proxy addresses, Azure AD Connect has a limit of 333 proxy addresses.

They do have a legitimate need for this account to receive mail for all of those addresses. We implemented a workaround by creating a group for the extra addresses. We removed 300 email addresses and put them on a group where that user is the only member. Mail flow is preserved and now both the user and the group can sync. The group is hidden from address lists to avoid confusing the users.

More information:

Tuesday, January 21, 2020

Reporting Script Duration

Currently working on a migration project where the source and target environments are quite large. We have a script that queries all mailboxes in the source and matches them to a target object.

The script takes 10-12 hours to run. We're making tweaks and want to see the effect, but we're not going to watch the script to verify the time to complete.

Here's a little bit of PowerShell that you can add to any script to measure the time to complete:

 #Start of Script  
 $start = Get-Date  
 
 
 #End of script  
 $end = Get-Date  
 
 # Calculate elapsed time  
 # Output in format hh:mm:ss  
 Write-Host “Script run time”  
 Write-Host $($end-$start)  

Friday, September 27, 2019

Azure AD Connect 1.4.x.0 Deletion Threshold Exceeded

Azure AD Connect is configured to perform automatic updates by default. When version 1.4.x.0 (in my case 1.4.18.0) is installed, device objects previously synced to Azure AD might be removed. Previous versions of Azure AD Connect synchronized devices that were not relevant. So, this release is cleaning them up.

For details, see:
In larger organizations, the number of devices deleted might be more than 500 which exceeds the deletion threshold. At this point, Azure AD Connect stops syncing. You might not notice it right away, but any new user accounts will not be synced up to Azure AD/Office 365.

In the Synchronization Service app, you will see a line with the status of:
stopped-deletion-threshold-exceeded

Before you attempt to fix the issues, you should verify that it is only device objects an not another accidental deletion issue. The steps for this from Microsoft are:

  1. Start Synchronization Service from the Start Menu.
  2. Go to Connectors.
  3. Select the Connector with type Azure Active Directory.
  4. Under Actions to the right, select Search Connector Space.
  5. In the pop-up under Scope, select Disconnected Since and pick a time in the past. Click Search. This page provides a view of all objects about to be deleted. By clicking each item, you can get additional information about the object. You can also click Column Setting to add additional attributes to be visible in the grid.

The fix for this issue is to allow the device deletes to occur by either increasing the threshold or disabling the threshold. You do this on your Azure AD Connect server using PowerShell.

To disable the threshold:
Disable-ADSyncExportDeletionThreshold
To increase the threshold:
Enable-ADSyncExportDeletionThreshold -DeletionThreshold 1000
To set the threshold back to default:
Enable-ADSyncExportDeletionThreshold -DeletionThreshold 500
 The Microsoft documentation about the deletion threshold is here:

 

Tuesday, September 17, 2019

Your administrator has blocked this application

I do a lot of work with Powershell and Office 365. To allow for multi-factor authentication when managing Exchange Online, you can use the Microsoft Exchange Online Powershell Module.

I installed the Microsoft Exchange Online Powershell Module on my computer some time back and had used it successfully. However, at some point it stopped working and gives the error:

Your administrator has blocked this application because it potentially poses a security risk. Your security settings do not allow this application to be installed on your computer.
 

For a while, I've been connecting with normal Powershell for management, but today I wanted to get this thing fixed. This error can apply to ClickOnce applications in general. It is not specific to the Microsoft Exchange Online Powershell Module.

There are trust levels that you can define for ClickOnce applications. These are set in HKLM\Software\Microsoft\.NETFramework\TrustManager\PromptingLevel. There are settings for different security zones. On my system, all of the zones were set to Disabled.



Valid values for these settings are:
  • Disabled. App is never allowed.
  • AuthenticodeRequired. User is prompted to allow app only if the app is digitally signed by using a trusted certificate that identifies the publisher.
  • Enabled. User is prompted to allow app even if not digitally signed.
I have set my zones to AuthenticodeRequired. Any app I'm using should be signed by the publisher and not self-signed. After making the change, I'm prompted to allow the app.



It seems odd that I needed to do this, but I have some Visual Studio components installed on this computer and that might have created the registry keys and set them to disabled.

Microsoft documentation for these registry keys is here:

Saturday, September 14, 2019

SAGE 50 Email Integration Woes

Sage 50 is a pretty common app in Canada for doing small business accounting. However, one of it's major drawbacks is really poor email integration. I think they've improved it somewhat in recent versions, but there is a MAPI dependency.

If you install the 64-bit version of Office, then Sage 50 will not be able to use Outlook to send messages. Now that 64-bit Office is the default for Office 365, you need to watch for that as step one. However, yesterday, on a new install of Sage 50, it wasn't working even with the 32-bit version of Outlook.

We got the error:
Sage 50 cannot communicate with your e-mail program. Please ensure that your email program is MAPI-compatible and that it is the default MAPI client

You also need to have Outlook configured as the default mail program. The Mail program in Windows was configured as the default. So, we changed that to Outlook. Still no luck. Same error.


The final fix for me was adding a registry key. According to a few people in discussion forums, without that key, Sage 50 doesn't load the mapi32.dll that's required to send email. Silly that this is still happening.

In the registry at HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows Messaging Subsystem, I needed to create a new String Value (Reg_SZ) named MAPI with a value of 1.

Some of the web pages I reviewed during troubleshooting: