Delete Leaf Objects from Active Directory User Object

The Story

The past days I had a colleague of mine come to me with a user migration problem. He wanted to migrate a user between two child domains in an AD forest. For this most of the time you use Microsoft’s ADMT (Active Directory Migration Tool). He went through the whole migration wizard and had the migration fail with an error message like this:

2014-03-17 12:47:27 ERR2:7422 Failed to move source object ‘CN=Joey’. hr=0x8007208c The operation cannot be performed because child objects exist. This operation can only be performed on a leaf object.

That was strange, I was expecting the user object to be a leaf object itself, not to contain leaf objects?! Then I remembered we are in 2014, we also use Microsoft Exchange and use ActiveSync on Mobile devices. In case you didn’t know, when you configure ActiveSync on your phone a special object is created under your User Object in Active Directory. This object is of type “msExchActiveSyncDevices” and list each of the mobile phones where you have configured Active Sync. I used adsiedit.msc to confirm that the leaf objects were indeed these msExchActiveSyncDevices objects.

So that explains what the leaf objects were, and how they got there. Now since the user is being migrated across domains, it really doesn’t matter whether those leaf AS objects are there or not, because after USMT users have to reconfigure their devices anyway, so they’re “safe to delete”. To fix this you can either use ADSIEDIT to locate the leaf objects and delete them or use Exchange Shell to delete the AD devices….or use Powershell to delete them from the user object just like you would with ADSIEDIT, which is what i want to share now.

The Script

I built this script on a Windows 8.1 computer with Powershell 4.0 and RSAT tools for Windows 2012 R2 installed. Also the AD environment is Windows 2008 R2 FFL and only Windows 2008 R2 DCs run on it. I didn’t check if this runs on less than this configuration, so do report back if this is not working on older combinations of RSAT + OS + AD, though i’m pretty sure you need at least one 2008 R2 DC in the user’s source domain, otherwise the powershell cmdlets won’t wor. You can download this script from the link: Delete-AS-ChildItems.

The script takes only one parameter the SamAccountName of the user with leaf objects. From then on it will show you the leaf objects it wants to delete, and then actually delete them, if not canceled.

Learning Points

I’ve made the script “autonomous”, in the sense that it will automatically discover the closest DC running AD Web Services, and query it for the SamaccountName. This snippet accomplishes that.

$LocalSite = (Get-ADDomainController -Discover).Site
$NewTargetGC = Get-ADDomainController -Discover -Service ADWS -SiteName $LocalSite
IF (!$NewTargetGC)
    { $NewTargetGC = Get-ADDomainController -Discover -Service ADWS -NextClosestSite }
$NewTargetGCHostName = $NewTargetGC.HostName
$LocalGC = "$NewTargetGCHostName" + ":3268"

Once we have this information, we query the GC for the SamAccountName and Domain Information. We need the domain to also discover the closest DC for that domain, and get the list of leaf objects (Lines 14-26)You will want to do this for 2 reasons: first because the GC partition doesn’t contain all the information you want (the child object information) and second, you can’t write to the GC partition, so you have to find your closest respective DC anyway.

The “trick” with this script is to use Get-ADObject with a search base of the User’s DN on a DC in the User’s Domain and look for the respective msExchActiveSyncDevices object type like below:

$UserOjbActSync = Get-ADObject -SearchBase $UserObj.DistinguishedName -filter { ObjectClass -like 'msExchActiveSyncDevices'} -Server $UserobjDC -ErrorAction:SilentlyContinue

Now to actually fix the problem, we run this command, that deletes all the child items, and whatever may be inside them.

Remove-ADObject $UserOjbActSync -Server $UserobjDC -Recursive:$true -Confirm:$false -Verbose -Debug

That about wraps this up. Just wait for replication to occur on your domain and you should e good to finish the migration. As always, use with caution, this does delete things. If you found this useful share it around 🙂

Quick Tip: Update Resource Records in Microsoft DNS using Powershell

One of the great things I like about the (not so) new Windows 2008 R2 Powershell modules is that we can now more easily manage the core Microsoft Networking services (DNS, DHCP). I want to share a little script I built that will add/update Host Records fed from a CSV file.

The Script

In the past automating this kind of thing was possible using a combination of WMI and VBS/Powershell and or batch scripting and using the famous DNSCMD. My script script will not work on any DNS server, you need to run Windows 2008 or later DNS, running against Windows 2003 DNS servers will yield strange/wrong results.

#sample csv file

#DNSName,IP,<other fields not used>
#foo.fabrikam.com,192.168.1.1,<other values, not used>

Param(
 [Parameter(Mandatory=$false)][System.String]$ResourceRecordFile = "C:\Temp\somefile.txt",
 [Parameter(Mandatory=$false)][System.String]$dnsserver = "DNS.constoso.com"
 )
import-module DNSServer

Write-Warning "This script updates DNS resource records in DNS based on information in a CSV file. Details are:`n
Using file $ResourceRecordFile as source file.`nMaking changes on DNS:$dnsserver`n
If you wish to cancel Press Ctrl+C,otherwise press Enter`n"
Read-Host

$HostRecordList = Import-csv $ResourceRecordFile

foreach ($dnshost in $HostRecordList) {
 $RR = $dnshost.DNSName.split(".")[0]
 $Zone = $dnshost.DNSName.Remove(0,$RR.length+1)
 [System.Net.IPAddress]$NewIP = [System.Net.IPAddress]($dnshost.IP)
 $OldObj = Get-DnsServerResourceRecord -Name $RR -ZoneName $Zone -RRType "A" -ComputerName $dnsserver -ErrorAction SilentlyContinue
 If ($OldObj -eq $null) {
 write-host -ForegroundColor Yellow "Object does not exist in DNS, creating entry now"
 Add-DnsServerResourceRecord -Name $RR -ZoneName $Zone -A -CreatePtr:$true -ComputerName $dnsserver -IPv4Address $NewIP
 }
 Else {
 $NewObj = Get-DnsServerResourceRecord -Name $RR -ZoneName $Zone -RRType "A" -ComputerName $dnsserver
 $NewObj.RecordData.Ipv4Address = $NewIP
 If ($NewObj -ne $OldObj) {
 write-host -ForegroundColor Yellow "Object to write different, making change in DNS"
 Set-DnsServerResourceRecord -NewInputObject $NewObj -OldInputObject $OldObj -ZoneName $Zone -ComputerName $dnsserver
 }
 }
 $OldObj = $null
 $NewObj = $null
 }

Learning Points

Running this script requires Windows 2008 R2 RSAT installed. As you can see, all the script needs is a CSV file with 2 columns called “hostname” and IP, containing the FQDN, and the DNS server you want to connect and make the changes.

Lines 17-18: This is where we’re extracting the short DNS name from the FQDN and the DNS zone name. Also we are converting the IP address to the format required for entry into DNS:

$RR = $dnshost.DNSName.split(".")[0]
$Zone = $dnshost.DNSName.Remove(0,$RR.length+1)
[System.Net.IPAddress]$NewIP = [System.Net.IPAddress]($dnshost.IP)

Lines 19-21: Here we try to resolve the DNS record, perhaps it already exists. We will use this information in the next lines…

$OldObj = Get-DnsServerResourceRecord -Name $RR -ZoneName $Zone -RRType "A" -ComputerName $dnsserver -ErrorAction SilentlyContinue

Lines 23:  To create a new Host record  (“A” type record). T he command is pretty straightforward:

Add-DnsServerResourceRecord -Name $RR -ZoneName $Zone -A -CreatePtr:$true -ComputerName $dnsserver -IPv4Address $NewIP

Lines 27-31: or To update an existing A record. No that there is a difference in how Set-DNSServerResourceRecord works compared to the ADD command. This one requires that we get the record, modify the IPV4Address field, then use it to replace the old object.

$NewObj = Get-DnsServerResourceRecord -Name $RR -ZoneName $Zone -RRType "A" -ComputerName $dnsserver
$NewObj.RecordData.Ipv4Address = $NewIP
If ($NewObj -ne $OldObj) {
write-host -ForegroundColor Yellow "Object to write different, making change in DNS"
Set-DnsServerResourceRecord -NewInputObject $NewObj -OldInputObject $OldObj -ZoneName $Zone -ComputerName $dnsserver
}

That’s about it. You can easily modify this script, so that you can pass the DNS server name from the CSV file (updating lots of records on multiple DNS servers) or updating multiple record type (A Records, CNAME Records). As always C&C is welcome.

Automate Replacing of Certificates in vCenter 5.1

A few days ago, VMware launched a much awaited tool, called SSL Certificate Automation Tool. This tool enables VMware administrators to automate the process by which they replace expired/self-signed certificates on all components of the VMware vCenter management suite. As many of you know this process, especially in the new v5.1 version is a complete pain to implement, error prone, and so many steps to follow that you are bound to make a mistake. Compared to say VMware vCenter 4.x, version 5.1 has more “standalone” components that need to interact with users or interact with each other to provide users with data/visualizations and must do that over secure connections.

The components I’m talking about are (the ones highlighted in orange are new to version 5.x vs 4.x)

  • InventoryService
  • SSO
  • vCenter
  • WebClient
  • LogBrowser
  • UpdateManager
  • Orchestrator

It might not look like much but it’s almost double the number of components, double the number of certificates and close to double the number of interactions between the components themselves. To make is “worse”, the workflow for replacing v4.x certificates  (I have this post where I automated the whole process for ESXi 4.x, extendable for vCenter 4.x) is different than the workflow for v5.x ones, much more complicated. Here‘s a document how to manually do it. But if you value your time read on, there’s  an easier way.

The relevant documentation for this automation process is available here from VMware. Essentially it a 2 step process:

Step1: Generate your certificates using OpenSSL and your Internal Windows Enterprise root CA, as per this document(Generating certificates for use with the VMware SSL Certificate Automation Tool (2044696)

Step2: Use the SSL Certificate Automation tool to deploy the certificates, as described here.

Automate Certificate Generation for use with the VMware SSL Certificate Automation Tool

I should say this again, your main source of information should be VMware’s KB article. The information here is either echoing that, or supplementing it where needed. Also things are presented in a different order in my post, so the whole process can be automated, unlike the KB which assumes manual work.

Prerequisites

1. The account you will use in this step must:

  • Be a Local administrator on the computer where the script presented will run
  • Be able to enroll certificates for the Certificate Template that will be used from your PKI infrastructure.
  • Be a Local Administrator on the servers where the vCenter components are installed.

2. Any commands/scripts presented here should be run from an elevated prompt.

3. Name resolution must work correctly on the client where this script will run (all vCenter components must be resolvable via DNS).

4. You must use the OpenSSLversion that VMware specifies, not newer, not older, that is openSSL 0.98 at the time of writing this post.

5. You need to have a certificate template configured according to VMware’s specifications, that means basically duplicating the default web server certificate of the Windows CA, with a few changes:

  • Go to Certificate Manager > Extensions > Key Usage > Allow encryption of user data
  • Also uncheck “allow private key to be exported” as the script will fail if the template allows this.

6. Your CA must have automatic approval activated so you can obtain the certificate using command line.

7. Obtain and save the certificates of your root CA and any intermediate CAs, as described in the KB. From the KB, as I understand it, you should name your root CA certificate root64.cer (x.509 format, base64 encoded). Any other intermediate or issuing CA certificates should be named root64-#.cer (x.509 format, base64 encoded), where # is a number starting from 1 to as many intermediates you have (where root64-1 is the intermediate closest to the issued certificate, and root64-N is the certificate closest to the root CA). Place all the certificates from the chain in a folder called “RootCA_chain“.

Note: Naming the files in a certain way is important. The KB does not do a good enough job of explaining how to build the .PEM file (either that, or I’m not doing a good enough job of understanding it), so make sure your CA chain certificates are numbered like this. if you don’t do this way you will have to rework the piece of code that sorts the certificate files for building the .PEM file.

8. Make yourself a custom openSSL configuration file. The configuration file must include only these lines, not more, not less (it’s a copy paste from VMware’s KB). Save this file as custom_openssl.cfg in the \bin directory where you installed OpenSSL.

[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = DNS:ServerShortName, IP:ServerIPAddress, DNS:server.contoso.com #examples only

[ req_distinguished_name ] # change these settings for your environment
countryName = US
stateOrProvinceName = Change State
localityName = Change City
0.organizationName = Change Company Name
organizationalUnitName = ChangeMe
commonName = Changeserver.contoso.com

9. Build a .csv file like this example below:

Name,DomainName,OUName
CNTvcS,contoso.com,vCenterServer
CNTvcO,contoso.com,VMwareOrchestrator
CNTvcIS,contoso.com,vCenterInventoryService
CNTVUM,contoso.com,VMwareUpdateManager
CNTwc,contoso.com,vCenterWebClient
CNTwc,contoso.com,,vCenterLogBrowser
CNTsso,contoso.com,vCenterSSO

As you can see, you need to specify a server name, domain name, and OU Name. The OU name is the name of the vCenter component, the value will be written in the “organizationalUnitName” part of the OpenSSl configuration file.

10. Create a folder (the script uses a static c:\temp\certs entry) where you will store all your certificates as they are generated. To this folder copy the RootCA_chain folder created above.

Just like with cooking you now have all the “ingredients” to generate your vCenter certificates.

The Script

You can download the script (Generate_vCenter_Certs_v5.1) from on my blog. My previous post on VMware certificates has a little more background information, that I won’t repeat here.

Learning Points

Lines 16-50: The script takes a few parameters, and also does some error checking on them.

    • The location of the openSSL directory ($OpenSSLPath)
    • The CSV from step 9, with all the servers and components of vcenter ($vCenterHostsFile)
    • The name\display name combination for your issuing/root Certification authority ($CAMachineName_CAName).
    • The name of the template to be used when issuing the certificates ($TemplateName).

Lines 52-71: More variables are set, and the script checks whether you have the root CA files in the specified location (default: c:\temp\certs\RootCA_chain).

Also this is where you should go in the script and change the Country, Company, State, and City Name. These will replace the values in the custom_openssl.cfg file.

Lines 76-78: Create a folder based on the Name-OUName combination to store all files used in certificate generation (CSR, private key, certificate, copy of the root CA chain certificates).

Lines 81-94: Customize the fields from the custom_openSSL.cfg file to match the specific server-service combination, according to the CSV file.

Lines 96-114:  We copy the template .cfg file to the working directory and replace each corresponding row with our values for name, city, state, etc. This is not written in the cleanest powershell for string manipulation, but it gets the job done.

Lines 116-131: This is where we generate the CSR to send to the CA. As per VMware’s instructions this is a 2 step process, first we generate the CSR, then we convert the private key to RSA format.

Lines 136-146: We use the windows tool certreq  to request a certificate and also retrieve the certificate. This is where being local admin in an elevated prompt comes into play, and also where automatic retrieval of certificates comes in handy.

Lines 151-159: We copy over the individual certificates of the Root/Intermediate/Issuing CAs to the working directory. Then we create a file called chain.pem, which will be the .PEM chained certificate file for the given server-component combination specified in the CSV.

Important!!! This is where reading the VMware KB wrong can cost you, and you won’t realize it until you run the SSL Automation Tool. I quote from the KB:

“Open the Root64.cer file in Notepad and paste the contents of the file into the chain.pem file right after the certificate section. Be sure that there is no whitespace in the file in between certificates.

Note: Complete this action for each intermediate certificate authority as well.”

This is where I initially made a mistake, and did it the wrong way.

  • I pasted the certificate in the .pem
  • I pasted the root after the certificate in the .pem
  • I pasted the intermediate/issuing CA certificates in the .pem after the root certificate

It is “do the same for each intermediate certificate” as in “sandwich your intermediate CAs certificates between the root and the certificate in descending order”, THE EXACT OPPOSITE of how Windows will display the certificate chain in the GUI. This is why I asked you in the first place to number your certificates in a specific way.

As a result, if you named your CA certificates like I explained above this code snippet will arrange them in the proper order and chain them in the .pem file correctly.

$chain_pem = "$rootdir\$($strHost.Name)-$($strHost.OUName)\chain.pem"
gc $crt | add-content $chain_pem</span>
$roots = dir "$rootdir\$($strHost.Name)-$($strHost.OUName)" root*.cer | sort name -desc
$roots | % {gc $_.FullName | add-content $chain_pem }

That wraps it up, needless to say, test this script before you let it run loose on your environment. Since the script has a lot of “Read-Host” in it, it is designed to run with pauses, to give you a chance to review the output, and cancel if there is a problem. After all you are running commands against your Certificate authority, so handle with care.

Once you have all the files you should follow step 2, and actually use the SSL automation tool.

Replace vCenter Certificates using the SSL Automation Tool

For reference here is VMware’s KB if you missed it above for using this tool. I don’t have much to say about this, other than stick to the document, but here are a few tips that will make your life easier:

  • If you have multiple servers running each service, it is best that you copy ALL the folders generated by the script (for each server-service combination) to each of the individual servers, and you can freely dispose of them once you successfully replaced the certificates.
  • Before you go ahead and copy over the Automation Tool, take the time to modify the SSL_Environment.bat file pointing each variable to its respective file/value. Then copy the SSL_Environment file to each server. This way each time you run the tool to update a certificate it will always know where to pick “inter-component-trust” certificates, user prompts and so on.
  • When you fill in the “set sso_admin_user=” variable put it in the format “user@domain” as it will give an invalid credentials error when you will run some steps.

I know it is a long read, but it is not an easy topic and I hope it helped you in your environment. Let me know in the comments if there is a way to improve this or an easier way to do this whole process.

Quick info #1 – Get list of all forest global catalogs

I decided to write some mini posts to remind myself the best way to get certain information from AD using Powershell (I find myself doing a lot of work around AD lately)

In most Active Directory forests all DCs are also GCs (global catalogs), there are very few use cases for not using GCs on all your DCs. This is what consultants from Microsoft doing AD Risk Assesments (AD RAP) will tell you: “Don’t think about it, just make all DCs, GCs”.

First time i wanted to get information about my DCs I used this syntax:

[system.directoryservices.activedirectory.Forest]::GetCurrentForest() | `
% {$_.DomainControllers} | select Name,Domain | `
Export-Csv c:\temp\AD\dcs.csv -UseCulture -NoTypeInformation

This essentially lists all your domain controllers in every domain. For 50DCs worldwide it takes around 2 minutes. However…while running this today in my environment I found that for a domain…the Domain Controllers Field was blank…I couldn’t tell you why, I’m still investigating, nevertheless I wanted to find another way to get this information. So I found this syntax, which is simpler, it gets the list of GCs:

[System.DirectoryServices.ActiveDirectory.Forest]::GetCurrentForest().GlobalCatalogs | `
select Name,Domain | sort-object Name,Domain | `
Export-Csv c:\temp\AD\dcs.csv -UseCulture -NoTypeInformation

As far as speed is concerned the time is just as long, so it is not faster, but this time I managed to get a list of all DCs, including the ones that did not show in the other object.