Updating and Verifying LDAPS Certificate

Windows domain controllers (DCs) can be used by applications as an LDAP server for user authentication or application data storage. To enable encrypted communication with LDAP on a DC, you need to install a certificate on the DC. This uses TLS similar to how a web server does for HTTPS.

If you have an enterprise CA in your Active Directory (AD) forest, a certificate is automatically issued to your DCs for encrypting LDAP communication. If you don't have an enterprise CA in your AD forest, then a certificate isn't issued to DCs automatically and LDAP communication is unencrypted.

Any server certificate added the the computer store on the DC is automatically used and available for secure LDAP. You don't need to configure the DC to use the certificate. You also have the option to install a certificate in the NT Directory Services store. If a certificate is installed into the NT Directory Services store, it is preferred over certificates in the computer store. This is useful when there are multiple services running on the DC (such as IIS) and you want to ensure that secure LDAP is using the correct certificate.

To update the certificate used by secure LDAP, put the new/renewed certificate in the same certificate store as the certificate that's being replaced. The newer certificate will be automatically selected and used for secure LDAP within a few minutes.

If you are a bit paranoid and want to confirm that the new certificate is being used before you remove the older certificate, there are no built-in Windows tools or logging events that display the information. You can use LDP.exe to test connectivity to port 636 and verify that secure LDAP is working, but this doesn't give you information about which certificate is being used.

However, you can download and use OpenSSL to verify which certificate is being used. OpenSSL can be installed on a client computer for testing connectivity rather than the DC. The following command connects to LDAPS on port 636 and displays information about the certificate being used to secure communication:

openssl s_client -connect DCName.domain.com:636

You can use the output from this command to verify the name in the certificate and expiration date.

Reference:

• https://learn.microsoft.com/en-us/troubleshoot/windows-server/identity/enable-ldap-over-ssl-3rd-certification-authority
• https://wiki.openssl.org/index.php/Binaries
  
• https://www.openssl.org/docs/man1.0.2/man1/openssl-s_client.html
  

 

AADSTS90072 User Does not Exist in Tenant

During a recent migration project from one tenant to another, a test user was unable to sign in. The sign-in page in Office 365 redirected to AD FS on-premises for authentication. The user credentials worked in AD FS and the web browser was redirected back to Office 365. Then this error was displayed: 

Sign in

Sorry, but we’re having trouble signing you in.

AADSTS90072: User account 'Bob.Smith@domain.com' from identity
provider'urn:sso.domain.com:domain.com' does not exist in
tenant 'Byron Co' and cannot access the application '4765445b-
32c6-49b0-83e6-1d93765276ca'(OfficeHome) in that tenant. The
account needs to be added as an external user in the tenant
first. Sign out and sign in again with a different Azure
Active Directory user account.

This error indicates that the user authenticated by AD FS does not exist in Azure AD. Based on the UPN, Bob.Smith@domain.com existed both in on-premises AD and Azure AD. So, there is some other property being used to match the two objects after AD FS authentication.

To understand where this broke down, you need to understand how objects in AD are linked with objects in Azure AD. There is an ImmutableID property on Azure AD users that links Azure AD users to on-premises AD users. Early implementations of Azure AD Connect (or Dirsync) copied the object GUID from on-premises AD and used that value for the ImmutableID. This worked well until you migrated user objects to new AD domain or AD forest where they'd have a different GUID.

New implementations of Azure AD Connect use ms-ds-ConsistencyGUID in the on-premises user object instead of GUID. This value can be copied between domains to preserve synchronization during object migrations. By default ms-ds-ConsistencyGUID is populated with the same value as the object GUID the first time the object is synced.

AD FS authentication uses the object GUID or ms-ds-ConsistencyGUID during authentication. This value must match the ImmutableID in AzureAD to allow authentication to complete. The older instructions for configuring AD FS authentication manually had you configure a rule for object GUID. If you use Azure AD Connect to configure AD FS then it creates rules that use ms-ds-ConsistencyGUID if populated or object GUID. This article talks about the rule configuration: https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-fed-management#modclaims.

If you update your deployment of Azure AD Connect to use ms-ds-ConsistencyGUID as the source anchor and forget to update AD FS to allow ms-ds-ConsistencyGUID in the authentication process, AD FS authentication will continue to work because object GUID and ms-ds-ConsistencyGIUD are the same value by default. However when you start migrating objects and retain the ms-ds-ConsistencyGUID (which will now be different from the object GUID) authentication starts to fail because the token passed back to Office 365 for authentication contains the object GUID which doesn't match the immutable ID/ms-ds-ConsistencyGUID. Thus the error message above.

In our case, the AD migration tool we were using copied the ms-ds-ConsistencyGUID from a source AD domain to target AD domain which caused our authentication issue. Because users were getting new mailboxes in this migration, we didn't need to maintain ms-ds-ConsistencyGUID. Our short term fix was to copy the object GUID value and place that in ms-ds-ConsistencyGUID and immutableID. However, the correct long term solution is to update AD FS to correctly use ms-ds-ConsistencyGUID during authentication.

This article has some examples you can use to convert object GUID to ms-ds-ConsistencyGUID and ImmutableID: http://byronwright.blogspot.com/2020/10/convert-immutableid-to-hex.html.

Script to Synchronize Primary Email Address with UPN

When planning an Office 365 implementation, it is best practice to start by assuming that UPN for signing in to Office 365 should match the user email address. If you don't configure it this way, then users have two separate items (their UPN for signing in and their email address) that look very similar. In many cases users are confused by the similarity.

If you are synchronizing  your on-premises Active Directory with Office 365 (in most cases you do) then you need to set the UPN for the on-premises user accounts with the correct UPN. The UPN from on-premises user accounts is synchronized to Office 365 to create the ID for signing in.

Most organizations are not using the UPN on user accounts for authentication on-premises. The option has been there since Windows 2000, but most organizations still use the domainname\username format for authentication. However, you need to verify if any user accounts are using the UPN for authentication before making this change. At minimum, you should communicate with your application and system administrators to see if they are aware of anything that might use UPNs. If your organization has issued certificates to users, they might be using UPN as the unique identifier for the certificate.

The script below does the following:

• Obtains a list of all users where the proxyAddresses attribute has a value. This is done so that the result include only user accounts with an Exchange attributes configured.
• Identifies the primary email address based on the all caps "SMTP:" text.
• Strips out the "SMTP:" text from the primary SMTP address.
• If the new UPN and the existing UPN do not match the user account is updated and the change is logged.

The location and name of the log file are configured in $logfile. You need to manually configure this variable and verify that the necessary folders exist.

  

 #Log folder must already exist  
 $logfile = "C:\Scripts\SyncUPN.txt"  
   
 #Adds timestamp to log file  
 Get-Date | Out-File -FilePath $logfile -Append  
   
 #Obtains only users with valid proxyAddresses attribute   
 $users = Get-ADUser -Properties proxyAddresses -Filter {proxyAddresses -like "*"}  
   
 #Prepare variables for processing status  
 $total = $users.count  
 $current = 0  
   
 Foreach ($u in $users) {  
   #Find primary SMTP address for user  
   $primarySMTP = $u.proxyAddresses | Where-Object {$_ -clike "SMTP:*"}  
       
   #Remove "SMTP:" to create the new UPN value  
   $newUPN = $primarySMTP.Substring(5)  
       
   #Set the new UPN value only if required  
   If ($u.UserPrincipalName -ne $newUPN) {  
     $u.DistinguishedName + " Old UPN: " + $u.UserPrincipalName | Out-File -FilePath $logfile -Append  
     $u.DistinguishedName + " New UPN: " + $newUPN | Out-File -FilePath $logfile -Append  
     Set-ADUser $u -UserPrincipalName $newUPN      
   } #end if  
   
   #Processing status  
   $current += 1  
   Write-Progress -Activity "Processing users to update UPN to primary email address" -Status "Progress: $current" -PercentComplete ($current/$total*100)   
        
 } #end foreach  

Script to Remove Old Domains from User Email Addresses

When managing email addresses and domains in Exchange Server, old email addresses are never removed automatically. This is good because it ensures that email addresses on a mailbox are never accidentally lost. However, you may want to clean up old domains or address formats that are no longer in use.

Some common scenarios where you might want to remove an old domain:

• An SMB deployment of Exchange Server where a .local domain was added as the first domain for email addresses.
• Old GroupWise addresses are left in place from an older migration.
• Obsolete domain left over from a company merger many years ago

I often find that obsolete domains are identified when I run IDFix as part of preparing to migrate to Office 365. To simplify the removal of obsolete domains, I have created the following script.

A few things to note:

• You need to set $RemovePattern to identify the domain to be removed. Any email addresses matching this pattern will be removed from proxyAddresses attribute in Active Directory objects.
• The script uses Get-ADObject rather than Get-ADUser to make sure that the domain is removed from distribution groups too.
• This version of the script is capable of removing multiple instances of a matching email address. So, if a user has several email addresses in the old domain, all of them are removed.
• At the end of the script, I use Write-Progress to display a status bar. It's not necessary, but if there is a large number of users it's nice to see activity on the screen instead of just waiting and hoping it's doing something.

 #This pattern is used to match the email addresses being removed.  
 #Test that this pattern finds the correct users and email addresses  
 #before running this script.  
 #Example: $pattern = "smtp:*@olddomain.com"  
 $RemovePattern = "smtp:*@olddomain.local"  
   
 Import-Module ActiveDirectory #only required for 2008 R2  
   
 #Get the users that have an email address that matches the pattern  
 Clear-Host  
 Write-Host "Querying objects...This may take a moment"  
 Write-Host ""  
 $objects = Get-ADObject -Filter {ProxyAddresses -like $RemovePattern} -Properties ProxyAddresses  
   
 #Identify address being removed from first user for warning  
 [String]$proxyexample = $objects[0].proxyaddresses -like $RemovePattern  
   
 #Display warning and get confirmation  
 Write-Host "You are going to remove email addresses that match the following pattern:"  
 Write-Host -ForegroundColor Red "$RemovePattern"  
 Write-Host ""  
 Write-Host "This is an example from the first object:"  
 Write-Host -ForegroundColor Red "$proxyexample"  
 Write-Host ""  
 Write-Host "This will modify $($objects.count) objects"  
 Write-Host ""  
   
 $confirm = Read-Host "Enter Y to continue"  
 If ($confirm -ne "Y") {Break}  
   
 #Prepare variables for processing status  
 $total = $objects.count  
 $current = 0  
   
 #Processing users to remove addresses  
 Foreach ($o in $objects) {  
     
   #Build list of addresses to remove for object  
   #required because there might be multiple that match  
   $proxy= New-Object System.Collections.ArrayList  
   
   Foreach ($a in ($o.ProxyAddresses)) {  
     If ($a -like $RemovePattern) {  
       $proxy.add($a) | Out-Null  
     } #end if  
   } #end foreach  
     
   #Remove each bad address  
   Foreach ($p in $proxy) {  
     Set-ADObject $o -Remove @{'proxyAddresses'=$p}  
   }  
     
   #Pause  
   
   #Processing status  
   $current += 1  
   Write-Progress -Activity "Removing email addresses that match pattern" -Status "Progress: $current" -PercentComplete ($current/$total*100)   
 } #end foreach  

Change All UPNs in a Domain

I needed to update all UPNs in a domain today. It was pretty quick to figure out, but here is one line to take care of it for you.

Get-ADUser -Filter * | ForEach-Object { Set-ADUser $_ -UserPrincipalName ($_.UserPrincipalName).Replace("OldDomain","NewDomain")}

Remember to make the pattern in the OldDomain unique enough that you don't accidentally change things you don't intend to. For example, if you are changing from a .local domain in the UPN to a .com, make sure that you replace ".local" and not "local" on the off chance one of the user IDs includes "local" in the name.

If there are any user accounts without a UPN, then an error is generated for those accounts. My domain had 4 accounts without a UPN:

• krbtgt - default account used for kerberos
• IWAM_ServerName - Old IIS account from Windows 2003
• IUSR_ServerName - Old IIS account from Windows 2003
• support_XXXXXXX - Used by Help and Support service

Unable to Start Data Collector Set

I was wanting to collect some performance information on a Windows 2008 R2 domain controller. One of the things you may not realize is that Windows Server includes some predefined data collector sets for common tasks and generates reports based on the data. Since I was having a performance issue on the DC, I wanted to run the Active Directory Diagnostics data collector set.

Built-in data collector sets

When I attempted to start the Active Directory Diagnostics data collector set (or any of the other predefined data collector sets), they didn't start. There was no error message or any indication of what the error might be.

Ultimately my workaround was to create a new data collector set with the same settings. Fortunately, when you create a new data collector set, one of the options is to create from a list of templates, which includes Active Directory Diagnostics. The new data collector set ran without any issues.

Selecting a template for a data collector set

SCOM AD Monitoring Alerts

I've been working with a larger client for the last several months on Active Directory (AD) issues. One of the ongoing small issues has been AD monitoring alerts generated in System Center Operations Manager (SCOM) when it appears nothing is actually wrong.

The alerts were appearing intermittently on several of the servers, but not all. We were seeing alerts like this:

Failed to ping or bind to the Infrastructure Master FSMO role holder
AD Op Master Response : The script 'AD Op Master Response' could not determine the schema Op Master.The error returned was: 'LDAP://DC.contoso.com/RootDSE' (0x8007203A)

Failed to ping or bind to the Schema Master FSMO role holder
AD Op Master Response : The script 'AD Op Master Response' could not determine the schema Op Master.The error returned was: 'LDAP://DC.contoso.com/RootDSE' (0x8007203A)

Failed to ping or bind to the RID Master FSMO role holder
AD Op Master Response : The script 'AD Op Master Response' could not determine the RID Op Master.The error returned was: 'LDAP://DC.contoso.com/RootDSE' (0x8007203A)

Failed to ping or bind to the PDC FSMO role holder
AD Op Master Response : The script 'AD Op Master Response' could not determine the PDC Op Master.The error returned was: 'LDAP://DC.contoso.com/RootDSE' (0x8007203A)

Failed to ping or bind to the Domain Naming Master FSMO role holder
AD Op Master Response : The script 'AD Op Master Response' could not determine the domain naming Op Master.The error returned was: 'LDAP://DC.contoso.com/RootDSE' (0x8007203A)

Script Based Test Failed to Complete
AD General Response : While running 'AD General Response' the following consecutive errors were encountered: Failed to bind to 'LDAP://DC.contoso.com/rootDSE'. This is an unexpected error. The error returned was 'LDAP://DC.contoso.com/rootDSE' (0x8007203A)

We also implemented AD Client monitoring and it showed the following errors:

Script Based Test Failed to Complete
AD Client Connectivity : The script 'AD Client Connectivity' failed while getting 'LDAP://DC.contoso.com/RootDSE.The error returned was: 'LDAP://DC.contoso.com/RootDSE' (0x8007203A)

Script Based Test Failed to Complete
AD Client Monitoring: AD Connectivity is unavailable, or the response is too slow. The bind to 'LDAP://DC.contoso.com/RootDSE' took 4259 milliseconds, which is longer than the allowed 1000 milliseconds.

After doing a packet sniff on one of the DCs, I was able to correlate a specific pattern of network activity with the failure to connect for AD Client Connectivity. When the connection error occurred the AD Client attempted to connect to the DC, but the DC refused the connection. In Network Monitor, this appears as:

• Client to server: TCP packet with the SYN flag set
• Server to client: TCP packet with Ack and Reset flags set

This behavior indicates that there is either no service listening on the port or that the service is refusing the connection. The service was definitely listening because it worked most of the time. So, the service must be refusing the connection for some reason. It wasn't a timeout of any type because the refusal was happening in about 1ms.

These domain controllers were running Windows Server 2008 R2. I did find some web pages, such as the one below, that indicated this type of error can be caused by IPv6 being disabled in Windows Server 2012:

• https://blogs.msdn.microsoft.com/nicole_welch/2015/02/monitoring-window-2012-ad-multiple-monitoring-scripts-failing-or-why-i-need-ipv6/

I can confirm that enabling IPv6 also fixed this issue for Windows Server 2008 R2. However, it didn't fix the problem immediately. If we had restarted the DCs, it probably would have taken effect immediately, but we enabled IPv6 during production hours and couldn't reboot right away. About 7 hours after IPv6 was enabled, all alerts related to the AD Client monitoring stopped and it's been all good since.

In our scenario, it was a little more difficult to be confident this would work because the only two servers experiencing this issue regularly were in a single location. That led us to think that there might be something odd in the network configuration there. Both of those servers were configured to use WINS for some legacy stuff in that site. I think that might have been an influencing factor somehow as that's the only configuration difference I could find.

Update:
The two servers having these errors had slow LDAP client bind times even after these errors went away. After a lot of monitoring, we added additional virtual processor to those two DCs. We increased from 2 vCPU to 4 vCPU. Once that was done, the slow LDAP client bind times disappeared. So, insufficient processing power seemed to play a part in this also. There were two DCs servicing a very large site (I think over 5000).

Interpreting RepAdmin.exe /ReplSummary

One of the basic tasks you can do to verify Active Directory replication health is to run RepAdmin.exe /ReplSummary. The question becomes, what exactly do the results mean?
If you’re looking for a quick analysis, here it is. With no fails and all largest deltas less than 1 hour, you’re all good.

Now, for a more detailed look…

Total is the number of replication connections that a domain controller has with other domain controllers. The number of connections is probably higher than you expect because a separate connection is created for each Active Directory partition that’s being replicated. Most of the time you will have 5 per domain controller (domain, configuration, schema, DomainDnsZones, ForestDnsZones).

Fails is the number of connections that are not replicating properly. The number of fails should be zero.

Largest delta is where some of the confusion comes in. This is the longest period of time that a connection has not been used between two domain controllers. So, if a domain controller has one connection that has not replicated for 45 minutes and the others have replicated in less, then this value is 45 minutes. That is why this value tends to be high.

Your other thought is likely: How can it take up to an hour for replication to occur?

Within an AD site, all changes are replicated within seconds. However, there are partitions that do not change often. In particular, the schema partition probably changes only every few months. However, even when there are no changes, the connection for a partition will communicate occasionally to verify that it didn’t miss a change notification (polling). The default value for polling within a site is once per hour. That’s where the up to 60 minutes comes from.

You can change the time limit for polling within a site to twice per hour or four times per hour. That will reduce largest delta value, but it won’t really make a difference in the replication of changed AD data. You’re just triggering that polling to happen more often. And, almost all of the time, polling doesn’t trigger any data replication because replication notifications within the site would have already triggered it.

If you do want to change this value, it’s done in the Schedule for the NTDS Settings object in the AD site. It can also be modified for each connection individually, but it’s preferred to do it at the site level.

Between sites, the schedule on the site links will determine how often the polling happens. Most organizations that I work with have dropped the schedule down to every 15 minutes from the default of 180 minutes. If it were left at the default of 180 minutes then largest delta could range up to 180 minutes.

Update Mount-ADDatabase for PowerShell v2

I'm working on some Active Directory (AD) disaster recovery projects right now and one of the recovery methods we're implementing is AD snapshots. With AD snapshots, you have a copy of your AD data to identify and recover from accidental changes.

The client I'm working with has Windows 2008 R2 with PowerShell 2.0 for their domain controllers. PowerShell is my preferred method for automating anything at this point but AD snapshots don't have any PowerShell cmdlets.

Fortunately Ashley McGlone, a Microsoft PFE, has created some PowerShell functions that help you manage and use AD snapshots. One of the coolest things in there is a function (Repair-ADAttribute) that lets you pull attributes from the snapshot and apply them to the same object in the production AD. You can read more about these functions and download them from these two locations:

• https://blogs.technet.microsoft.com/ashleymcglone/2014/04/24/oh-snap-active-directory-attribute-recovery-with-powershell/
• https://gallery.technet.microsoft.com/Active-Directory-Attribute-0f815689

The minor issue I ran into is with the Mount-ADDatabase function. This function has a -Filter parameter which displays a list AD snapshots and lets you choose which one to mount. In Ashley's function this is done by using Out-GridView with the -OutputMode parameter which requires PowerShell v3. Using Out-GridView is an easy way to allow the user to select the snapshot. I wish it worked for my servers using PowerShell v2. Here is the line from the function:

 $Choice = $snaps | Select-String -SimpleMatch '/' |  
       Select-Object -ExpandProperty Line |  
       Out-GridView -Title 'Select the snapshot to mount' -OutputMode Single  

For my project, getting all of the DCs upgraded to using PowerShell v3 would take a while. I also didn't want to leave the project in a place where a whole bunch of manual steps were required to mount an AD snapshot older than the previous day. So, let's convert this to a method that works in PowerShell v2.

Now I needed a way to convert a list of snapshots in to a menu. My starting point was a TechNet discussion posting from Grant Ward (Bigteddy). You can view his solution for a discussion here:

• https://social.technet.microsoft.com/Forums/scriptcenter/en-US/74c34d41-1cf6-494e-8fe2-13827c3b7a4d/create-dynamic-menu-system-with-powershell?forum=ITCG

Using that example I created the following code:

 $choices = $snaps | Select-String -SimpleMatch '/' | Select-Object -ExpandProperty Line  
 $menu = @{}  
 $i=0  
 foreach ($s in $choices) {  
      $i++  
      Write-Host "$i --> $s"  
      $menu.Add($i,$s)  
 }  
   
 [int]$ans = Read-Host 'Enter selection'  
 $Choice = $menu.Item($ans)  

This code takes the list of snapshots in the variable $snaps and does two things:

• Writes a menu to the screen
• Add each menu item to the array $menu

After the menu is displayed on the screen and the user selects an option (the $ans variable), the option is used to place the snapshot name into the $Choice variable for further processing. Now we have a version that works in PowerShell v2.

Finding the User or Group Name from a SID

I'm working on project where we needed to set AD security permissions in a test environment based on the permission based on production. When I generated a report of AD permissions that had been applied, several of the entries came back with SID numbers instead of user or group names. Typically this means that the user or group has been deleted, but I wanted to confirm.

I wanted to take the SID and identify the user or group account that was associated with it. After a quick search I found a few examples that looked similar to this:

 $objSID = New-Object System.Security.Principal.SecurityIdentifier("S-1-5-21-1454471165-1004335555-1606985555-5555")  
 $objUser = $objSID.Translate([System.Security.Principal.NTAccount])  
 $objUser.Value  

Above example taken from: https://technet.microsoft.com/en-us/library/ff730940.aspx

It seemed to me that there had to be an easier way using the ActiveDirectory module for PowerShell which isn't used by these examples. Good news, there is!

You can't use Get-ADUser or Get-ADGroup to identify the SID name because it could be either one. However, you can use Get-ADObject:

 Get-ADObject -Filter {objectSID -eq "S-1-5-21-1454471165-1004335555-1606985555-5555"}  

If the command does not return any results then there is no AD object with that SID.