2010 and 2011 were virtualization years for me, I worked on several projects in design, implementation, and I learned so much, that looking back I really get a feeling of accomplishment.
I’ve also been a little “cutting edge”, non conservative with my designs some would say. I guess practice what you preach kind of stuck with me and I made it my mission to build reliable, self contained VMware environments, as much as possible.

As part of the design process, you always have to think about your management software

• Where do you put the pieces of software that help you manage the environment?
• How do you ensure availability and SLA for these components to allow you to recover from failures?

The answer to the first question can be:

Option A: In a management cluster, dedicated to management software for the virtualization stack

• The advantage is you always know where the VMs are, if you have a failure there are 2 servers they can be on.
• The disadvantage is you dedicate two physical boxes for this purpose, which can have a maximum utilization of around 40% for failover reasons.

Option B: Next to production machines

• The advantage is you don’t have to setup a management cluster, and you optimize resource utilization in your datacenter.
• The disadvantage is that you lose “determinism”, the security of “I know on what server vCenter is sitting, so i don’t have to look for it”, if i get a cluster failure or worse.

Well I’ve come up with two “tricks” that tackle the drawbacks of the second option, not knowing where your management servers are, making it a preferred choice if your environment does not warrant a dedicated management cluster just for that.

#1 Track the movement of vCenter and vCenter DB using vCenter Alarms

This one is a really easy way to keep track of your vCenter components. It works best combined with the second trick you will see below, mainly because it does not cover all scenarios but the advantage of this method is that the information is provided in real time.

What I am proposing is that you create an alarm in vCenter, that monitors for events that change the VMhost of your vCenter VM. These events are:

1. VM is being migrated (manually)
2. VM is being migrated by DRS
3. VM is being restarted by HA on another host

The third trigger will be hit and miss, it stands to reason, that if vCenter is not up to send the mail since it being restarted, you may or may not get the email, or you will get it after the fact. nevertheless it is good to have it there.

Below are the screenshots of how the alarm would look like:

On the advanced field put this condition in:

Then add some notification address or whatever you prefer

Save your alarm, and then try to migrate vCenter and see what happens. You should do this to the vCenter DB server aswell, and any components you feel you should know where they are, for troubleshooting purposes (VUM, Nexus 1000v Supervisor Modules, Management Appliances).

#2 Check the vSphere host where vCenter is running using a scheduled script

Another wasy to check where your vCenter components stay is using a scheduled PowerCLI script that runs once a day and sends you an email where vCenter VM and vCenter database are sitting (which vSphere host)

This script assumes following:

1. vCenter VM name in inventory = vCenter VM hostname
2. vCenter is using separate database, if you don’t care about that, you can remove the references to the DB.
3. vCenter Database name in inventory = vCenter Database hostname or at least a cNAME with this name (e.g. RO-vcenter > RO-vCenter-DB name, and alias in DNS)

You can customize this by entering a CSV file of the names of the vcenter instances and their respective databases.

 #version 0.1
#initial release

Add-PSSnapin Vmware.VimAutomation.Core -ErrorAction:SilentlyContinue
Set-PowerCLIConfiguration -DefaultVIServerMode multiple -Confirm:$false
#Write-Host -ForegroundColor Yellow "This script Generates a report detailing which host has the vCenter VM and vCenter DB VM`
#If you wish to cancel Press Ctrl+C,otherwise press Enter"
#Read-Host

#using fqdn because certificates are issued using a FQDN
$vCenter = ('vcenter','vcenter2','vcenter3')

If ($global:DefaultVIServers -ne $null) {
	DisConnect-VIServer * -Force -Confirm:$false }
$vCenter | % {Connect-VIServer $_ -NotDefault:$false}

$Report = @()
$vCenters = $global:DefaultVIServers | % {
	$row = "" | select vCenterInstance,FrontendVMHost,DBVMName,DBVMHost
	$row.vCenterInstance = $_.Name
	$row.FrontendVMHost = (get-vm -Name $_.Name.Split(".")[0] -server $_.Name).VMHost
	#db is hostname + db
	$dbvm = "$($_.Name.Split(".")[0])DB"
	$DBVMName = ([System.Net.Dns]::GetHostByName($dbvm)).HostName.Split(".")[0]
	$row.DBVMName = $DBVMName
	$row.DBVMHost = (get-vm -Name $DBVMName* -server $_.Name).VMHost
	$Report += $row
}

$FileDate = get-date -Uformat "%Y%m%d-%H%M%S"
$Path = "c:\temp\vsphere\"
$File = "$FileDate-vCenter-InfraLocation.csv"
$Report | Export-Csv -NoTypeInformation -UseCulture "$Path$File"

$encoding = [System.Text.Encoding]::UTF8
#I made the convoluted out-string construct because the object cannot be serialized"
$ReportBody = $null
$ReportBody += $Report | % { "
`n
`n$($_.vCenterInstance)`n$($_.FrontendVMHost)`n$($_.DBVMName)`n$($_.DBVMHost)"}$Body = "</pre>
<div>I'm the PowerCLI Magic Script. This is the list of your vCenter instances and their locations in the Infrastructure.
`
If you ever lose track of them, this email is the reminder. The latest update is from $FileDate
`
Below is the detailed information about each Instance:
`
`n`n`n`n`n`n$ReportBody`n`n
<table border="`&quot;1`&quot;">
<tbody>
<tr>
<td>vCenterInstance</td>
<td>FrontendVMHost</td>
<td>DBVMName</td>
<td>DBVMHost</td>
</tr>
</tbody>
</table>
</div>
<div>"

Send-MailMessage -Smtpserver smtpserver -From 'admin_vmware@foo.com' -To 'vSphereAdministrators@foo.com' -Body $Body -Bodyashtml -Encoding $encoding -Subject "vCenter Instances List" -Attachments $Path$File

Learning points:

Line 11: This is where you define you vCenter server names, if you have more of them. I had 3 for example.

Line 22 & 27: This is where you perform a get-vm to find out the host where this VM is residing on

The rest of the script is just to cycle through all vCenter instances and create an email that it sends to a given email address.

Perhaps to some people this may seem unnecessary, as they may not have faced major outages, perhaps to some it may seem that these monitoring tricks are not enough to cover monitoring of all ‘outages’ situations, but I find it is not worse than having a separate management cluster, with the added benefit of not having to deal with another separate management cluster.

C&C as always is welcome

$InputObject = Read-host -Prompt "Insert Computername to get list of installed updates";
$Report = @()
$filename = "$env:Temp\Report_$(get-date -Uformat "%Y%m%d-%H%M%S").csv"
$InputObject | % {
   $objSession = [activator]::CreateInstance([type]::GetTypeFromProgID("Microsoft.Update.Session",$_))
   $objSearcher= $objSession.CreateUpdateSearcher()
   $HistoryCount = objSearcher.GetTotalHistoryCount
   $colSucessHistory = $objSearcher.QueryHistory(0, $HistoryCount)
   Foreach($objEntry in $colSucessHistory | where {$_.ResultCode -eq '2'}) {
       $pso = "" | select Computer,Title,Date
       $pso.Title = $objEntry.Title
       $pso.Date = $objEntry.Date
       $pso.computer = $_
       $Report += $pso
       }
   $objSession = $null
   }
}
$Report | where { $_.Title -notlike 'Definition Update*'} | Export-Csv $filename -NoTypeInformation -UseCulture
ii $filename

Once you run this report and have an csv viewer installed (excel for example( it will open up the file so you can review it. When exporting I did a filter to remove MS Forefront definition updates as it is pretty irrelevant most of the time, you use other tools to manage Forefront definitions.

Learning Points

Line 05 – This line creates and instance of the Windows Update API. What is neat about this function is the fact that can create an instance of the API and connect to a remote computer, notice the “$_” at the end of the line.

Line 09 – In this line after searching the entire history we filter out all but successful updates. Yes it would be nice to do that in the actual search, but I don’t know if it is possible. So I resorted to filtering out only successful result codes.

Below is a table with possible values. This can be useful if you want to generate a report based on the result code

That’s about it with getting the list of installed updates, the bit of code above can be easily integrated to run across a large number of computers. Thanks for reading and feedback.

Here is what you will need to accomplish this:

• Exchange Server Administrator Rights where the RSGs are located.
• Exchange Management Shell since this entire procedure is best done using powershell scripting.
• Run the script as Administrator to avoid errors due to enabled UAC

The Script

#Dismount DB, Remove, DB, Remove Storage Group
$DB_in_RSG = Get-MailboxDatabase -Server <ENTERSERVER> | where {$_.Recovery -like '*true*'}
Write-Host -ForegroundColor Green "The Database in the RSG ($($DB_in_RSG.Name)) will be Dismounted. Press Ctrl+C to cancel or Enter to continue"
Read-Host
$DB_in_RSG | Dismount-Database -Verbose -Debug

Write-Host -ForegroundColor Green "The Database in the RSG will be Removed. Press Ctrl+C to cancel or Enter to continue"
Read-Host
$DB_in_RSG | Remove-MailboxDatabase -Verbose -Debug

#Now deleting the files of the database
$SG_Path = (Get-StorageGroup $DB_in_RSG.StorageGroup).SystemFolderPath
Write-Host -ForegroundColor Green "The actual DB files will be removed (stored in $SG_Path). Please check previous steps completed successfully before continuing. Press Ctrl+C to cancel or Enter to continue."
Read-Host
get-item $SG_Path | del -Force -Recurse -Verbose -Debug

#Now removing storage group
Write-Host -ForegroundColor Green "The RSG will be Removed. Press Ctrl+C to cancel or Enter to continue"
Read-Host
Remove-StorageGroup -Identity $DB_in_RSG.StorageGroup -Verbose –Debug

Learning Points

There are a few steps you need to do:

1. Grab the RSG (there is only one per server) – stored in the script in $DB_in_RSG
2. You dismount the database from the server using Dismount-Database cmdlet.
3. After disomunting the database you remove the Database using Remove-MailboxDatabase cmdlet. This will just remove the DB from Exchange, the files will remain on the file system.
4. The actual files on the file system get removed using the path from variable $SG_Path. Use del cmdlet with -force -recurse to bypass any confirmation prompts.
5. The last step is removing the Recovery Storage Group from the Exchange. The commandlet for this is Remove-StorageGroup.

That’s about it, pretty easy actually. Although this script is the last in the series I reccomend you actually run this before anything, before mounting the RSG or at least do a check to see if a RSG is already mounted.

Here is what you will need to accomplish this:

• Exchange Server Administrator Rights where the mailbox(es) are located.
• Full mailbox access for exporting emails out of the temp mailbox I was talking about
• Exchange Management Shell since this entire procedure is best done using powershell scripting.
• Obviously free space both on the Database disk aswell as disk space where the PST file will be saved.

The Script

#Parameters Section
param (
 [parameter(Mandatory = $true)]
 [string]$paramRestoredUser
)

cls
#Restore mailbox section
$Filter = "SamAccountName -like '$paramRestoredUser'"
$SourceAlias = Get-Mailbox -Filter $Filter -IgnoreDefaultScope

If ($SourceAlias -eq $null) {
 Write-Host -foregroundcolor Red "No Mailbox for SamAccountName $paramRestoredUser found.Script will quit"
exit
 }

$RSG_DB = Get-MailboxDatabase -Server $SourceAlias.ServerName | where {$_.Recovery -like '*true*'}
if ($RSG_DB -eq $null) {
 Write-Host -ForegroundColor Red "No RSG was found on $($SourceAlias.ServerName). Script will quit now!"
 exit }

$TargetAlias = Get-Mailbox e2k7_Restore_MBX
Write-Host -ForegroundColor Green "Input the date of the restored mailbox:"
$RestoreDate = Read-Host

$TargetFolder = "$($SourceAlias)_$RestoreDate"
Restore-Mailbox -Identity $TargetAlias.Alias -RSGMailbox $SourceAlias.ExchangeGuid -RSGDatabase $RSG_DB -TargetFolder $TargetFolder -BadItemLimit 1000 -Verbose -Debug -Confirm -ValidateOnly

echo "-ValidateOnly switch from the command above is removed. Please check no errors occured above and everything is configured properly. Press Enter to continue CTRL+C to cancel the script!"
Read-Host
Restore-Mailbox -Identity $TargetAlias.Alias -RSGMailbox $SourceAlias.ExchangeGuid -RSGDatabase $RSG_DB -TargetFolder $TargetFolder -BadItemLimit 1000 -Verbose -Debug -Confirm

#building an excluded folders list, just in case we did another restore and the previous content was not deleted
$IncludedFolders = $TargetFolder
$ExclFoldersList = @()
$ExcludedFolders = Get-MailboxFolderStatistics $TargetAlias | where-object { $_.FolderPath -notlike "*$($IncludedFolders)*"} | Select-Object FolderPath | foreach-object {
 $ExclFoldersList += $_.FolderPath }
$ExclFoldersList = ([string]::join(",",$ExclFoldersList)).Replace("/","\")

$UserName = $TargetAlias
$PSTPath = "Q:\UserPersonalFolders"
"Excluded folders list: $ExclFoldersList"
Read-Host
export-Mailbox -Identity $UserName.SamAccountName -BadItemLimit 1000 -DeleteContent $True -PSTFolderPath $PSTPath -ExcludeFolders $ExclFoldersList

The details for the Script

First the script gathers data about the restored mailbox, and from that info the Recovery Storage Group ($SourceAlias). It also needs a target mailbox, where the restored data will be stored ($TargetAlias) (needless to say I did in a “forest friendly way, using -Filter cmdlet looking up the given SamAccountName). The data will be moved from the mailbox in the RSG into the $TargetAlias mailbox. It will be placed in a folder, named with the date of the restored data ($RestoreDate). Restoring the data is done with this command:

Restore-Mailbox -Identity $TargetAlias.Alias -RSGMailbox $SourceAlias.ExchangeGuid -RSGDatabase $RSG_DB -TargetFolder $TargetFolder -BadItemLimit 1000 -Verbose -Debug -Confirm -ValidateOnly

Now we want to export the data from the $TargetAlias into a PST file. This is easy to do via the Export-Mailbox cmdlet. The problem appears when you do for instance multiple exports in the same target mailbox and then a mass export, or just have left old data in the target mailbox. You do get data separated into folders, if your $TargetFolder name is unique, but the Export-Mailbox cmdlet cannot export data based on this information (it can only do some filtering (date,content,subject) and I believe it is resource intensive). What Export-Mailbox does have is the “-Excludefolders” parameter which lets you not export certain data.

The trick I came up with was to scan the $TargetAlias mailbox for foldernames (I used Get-MailboxFolderStatistics for that) and build a list of the folders that did not contain the value I entered in the $TargetFolder variable upon restore. Besides that you also have to do some parsing of the output from get mailboxfolderstatistics into something Export-Mailbox understands. This is what I did:

$IncludedFolders = $TargetFolder
$ExclFoldersList = @()
$ExcludedFolders = Get-MailboxFolderStatistics $TargetAlias | where-object { $_.FolderPath -notlike "*$($IncludedFolders)*"} | Select-Object FolderPath | foreach-object {
 $ExclFoldersList += $_.FolderPath }
$ExclFoldersList = ([string]::join(",",$ExclFoldersList)).Replace("/","\")

In the end I just ran the Export-Mailbox on the target mailbox, specifying the excluded files and that was that.

The whole process is pretty easy, the actual restore is done in a one liner (Restore-Mailbox), but as you can see error checking and failproofing the script make up the rest of the work. Hope you enjoy this, next up is how to dismount a RSG, the scripted “clean-way”.

Q: But why don’t we use the nice GUI Tool from Exchange Management Console (Extra.exe) and do it from there, “we don’t need no scripting”?

A: My experience tells me the scripted method is safer and works “as expected” unlike the GUI, which says it did something, when it didn’t (I’ve spent days trying to figure out why a RSG Database won’t actually dismount when the GUI said: “Completed Successfully”.

OK, let’s get on with it. All that I am about to explain requires Exchange Administrator privileges on the Exchange servers.

We will be creating a Recovery Storage Group, this is the first step in the restore process. To create the RSG you need following:

• Adequate disk-space to restore the mailbox database, locally on the Exchange Server where the DB was residing
• Exchange Management Shell running as Administrator (especially on CCR clusters)
• No other Recovery Storage Group already created on that server with an existing RSG database (you can only have 1 RSG with 1 DB in the RSG). It is best to remove any previous RSG completely then recreate it for your needs.
• Specific information like which DB to link to the RSG and the list of mailboxes to restore.

Creating a Recovery Storage Group can be as easy as this:

New-StorageGroup -Server <MBX Role Server Name> -Name <StorageGroup Name> -LogFolderPath <Logs Folder> -SystemFolderPath <SystemFiles Path> -Recovery -Verbose

The command is very similar to creating a new SG, except for the -Recovery switch, designating it as a Recovery Storage Group. I added the -Verbose switch so you can see what is going on behind the scenes.

New-MailboxDatabase -MailboxDatabaseToRecover <Mailbox Name> -StorageGroup <Recovery Storage Group Name> -EdbFilePath <path to store edb file> -Verbose

Here it is just as easy as creating a new mailbox database, only you are creating it in the recovery storage group you created with the previous command. The key thing to remember here is that the value of the “MailboxDatabaseToRecoverParameter” must be the exact same name of the mailboxDB of which you want to recover from. If the name is different you will not be able to run any restore commands, because it will not be able to find any mailboxes when it searches the recovered database.

A working script for creating the RSG

Below I’m sharing with you a working snippet that should help in creating a recovery storage group and DB. In short here is what the code does:

Using a given UserPrincipalName…

• Attempts to retrieve the mailbox for the UPN (it is a “forest friendly” coding for retrieving the mailbox). If it fails it quits
• Checks if a folder structure for placing, logs, system files and the edb file exists (I used a location called d:, use a variable if you like).
• If folders already exist, it will quit, otherwise it will create a folder with the MDB name, and logs and edb subfolders,
• Next it checks if a Recovery Storage Group already exists, unless you cancel the script it will continue to use this RSG, with the given details. Otherwise it will create a RSG on its own.
• It will then create a mailbox database where you / your backup admin will restore your exchange backup.

$MBX_UPN = Read-Host
$Filter = "UserPrincipalName -like '$MBX_UPN'"
$SourceMBX =  get-mailbox -IgnoreDefaultScope -Filter $Filter
If ($SourceMBX -eq $null) {
	Write-Host -foregroundcolor Red "No Mailbox for $MBX_ID found`nScript will Quit"
	exit }
Write-Host -ForegroundColor Green "Source Mailbox is`n $SourceMBX"

$LinkedMDB = Get-MailboxDatabase -Identity $SourceMBX.Database
Write-Host -ForegroundColor Green "Ok, Database ($($LinkedMDB.StorageGroup.Name)) is grabbed, now creating RSG Folders and RSG`nPress Enter to continue or Ctrl+C to Cancel"
Read-Host

#Checking if the RSG folders already exist, if not attempt to create them
If ((Test-Path "d:\$($LinkedMDB.StorageGroup.Name)")) {
 Write-Host -ForegroundColor Red "Folder already exists. Please remove d:\$($LinkedMDB.StorageGroup.Name) before running this script again.`nScript will quit"
 exit
 }
$SysPath = New-Item -Type Directory -Path d: -Name $LinkedMDB.StorageGroup.Name | Get-Item
If ((Test-path $SysPath)) {
 $DBPath = New-Item -Type Directory -Path $SysPath -Name DB | Get-Item
 $LogsPath = New-Item -Type Directory -Path $SysPath -Name Logs | Get-Item
 }
#If folders were created successfully we can continue
If ((Test-path $SysPath) -and (Test-path $DBPath) -and (Test-path $LogsPath)) {
 #Checking if RSG already exists
$RSG_check = Get-StorageGroup -Server $LinkedMDB.ServerName | where {$_.Recovery -like "True"}
 If ($RSG_check -ne $null) {
 Write-Host -ForegroundColor Magenta "A RSG was found on $($RSG_check.ServerName). Here are RSG Details:"
 $RSG_Check | select-object Name,Identity,Recovery,LogFolderPath,SystemFolderPath | fl
 Write-Host -ForegroundColor Magenta "To use this RSG Press Enter, to cancel Press Ctrl+C"
 Read-Host
 }
 Else {
 Write-Host -ForegroundColor Green "Now creating Recovery Storage Group..."
 New-StorageGroup -Server $LinkedMDB.Server -Name "Recovery Storage Group" -LogFolderPath $LogsPath.FullName -SystemFolderPath $SysPath.FullName -Recovery -Verbose
 }
 Write-Host -ForegroundColor Green "OK! No RSG found. Now creating RSG Database..."
 New-MailboxDatabase -MailboxDatabaseToRecover $LinkedMDB.AdminDisplayName -StorageGroup "$($LinkedMDB.ServerName)\Recovery Storage Group" -EdbFilePath "$($DBPath.FullName)\$($LinkedMDB.Name).edb" -Verbose
 }
Else {
 Write-Host -ForegroundColor Red "Could not Create folder or folder structure in d:\$($LinkedMDB.StorageGroup.Name). Check messages above for errors! Script will quit."
 exit
 }

This is about it with creating a Recovery Storage Group, it is actually not difficult, just remember to name the MDB inside the RSG with the same name as the source MDB (this was also required on Exchange 2003, as far as I know). Also you cannot have more than one RSG per Maibox Server, it is best to remove any RSG you have after you are finished recovering data. Next post we will discuss how to restore data from a MDB and how to remove the RSG.

As always I value your feedback and hope you found this post useful.

“Energy” Related WMI Classes

Here are a few interesting classes I stumbled upon. Some of them are only available under Windows7 probably also Vista, but I’m not sure.

• WmiMonitorBrightness – gives information about monitor brightness. For example these line give the max. “value” and current value of brightness

$MaxBrightness = get-wmiobject -class WmiMonitorBrightness -Namespace root/wmi).level | measure-object -Maximum).maximum
$CrtBrightness = "{0:P0}" -f ((get-wmiobject -class WmiMonitorBrightness -Namespace root/wmi).CurrentBrightness/$MaxBrightness)

• Win32_PowerPlan – provides information and identifiers about the powerplans defined. In this class ALL powerplans are defined, and just the active plan has an “IsActive” flag attached it, here’s how to get it:

$powerplan = (Get-WmiObject -Class win32_powerplan -Namespace 'root/cimv2/power' | where {$_.IsActive -eq $true}).ElementName

• Win32_Processor – gets information about the CPU (I was interested in the CPU load for statistical purposes). This one was pretty easy to find, the value was written in plain sight. Take a look:

$cpu = (Get-WmiObject Win32_Processor).LoadPercentage

• Win32_Battery – Provides information about the battery itself (estimated time, remaining load, power status). Running “Get-WmiObject -Class Win32_Battery | gm” take a closer look at these members:
  • BatteryStatus – this will toggle between ’1′ meaning on Battery and ’2′ meaning on AC Power
  • EstimatedRuntime – this will be the number of minutes running on battery, as the OS estimates it, and if you get a very high value (tens of thousands) when you plug the AC Power, it means the battery is charging
  • EstimatedChargeRemaining – percentage-wise representation of battery charge remaining

Powershell + Event-Log “101″

I used this battery and power experiment to learn more about working and writing data to the Event-Log. I wanted to create a new “Event-Log” in Windows (windows 7 as you probably know allows for a lot of application logs) and then write events to it. Then at any point you can export the Event-log to csv. The following creates an Event-Log, with the name “BatteryMonitor” from the Information category (for my uses “Source” was not needed but it is a required parameter:

New-EventLog -Source BattMon -LogName BatteryMonitor -CategoryResourceFile Information

You can also check if an Event-log is created exists you can use this scriptlet (the answer lies in WMI this time, I didn’t find a cmdlet that does it faster):

(get-wmiobject -class "Win32_NTEventlogFile" | where {$_.LogFileName -like 'BatteryMonitor'} | measure-object ).count -eq '1'

Finally here’s how to write to the event-log, a new event. This bit I used in a script to mark the execution of the script in the event-log:

Write-EventLog -LogName BatteryMonitor -Source BattMon -EventID 65533 -Message 'Starting new Execution of BatteryCharge Monitor Script. The script will pump here CSV values. Values are listed in this order, as CSV: PowerPlan,PowStatMsg,ChargeRemMsg,RemTimeMsg,RAM,CPU,CrtBrightness' -EntryType Information -ComputerName $env:computername -ErrorAction:SilentlyContinue

So that is about it, as usual I tried to tie all of these scriptlets into a usable script, you can download it from here.

Essentially you can get some info about the message and mailboxes, and use it with Export-mailbox to remove the data. That is how I initially found this link, but what is not written there is that you need to have all the prerequisites for running export-mailbox, and also running it on hundreds of mailboxes may take a while. I decided to do it my way,by building on what I found on that blog.

This is “Mass Remove message(s) from mailboxes – My Way”.Depending on your situation you can apply these steps multiple times:

1. Identify the message that started it all
2. Track the message on the Exchange Servers and compile a list of unique recipients of the message.
3. Remove the message from the offended mailboxes (there may be special requirements to perform the task, see here)

Identify Message

Getting the information should be pretty easy, someone probably forwarded you the copy of the message(s) to be dealt with. You want to get this info as a minimum: subject,sender,date and time message(s) was/were sent. When you have enough info, open the Exchange Management Console > Tools > Message Tracking and from there identify which of the events represent the time the message originally arrived on the Transport Servers. For that event grab the “MessageID” Value. We will use this in the following steps to find all events relating to that specific messageID.

Track Message

Assuming the worst case scenario you have to do tracking across all Exchange Transport Servers, the speed of the process depends on how close to your Exchange Transport Role Servers you are running the tracking. I suggest you make sure this process runs in the same LAN as the Exchange, especially the export-mailbox part. Anyhow, to get all messages sent by “baduser@foo.com” across all transport servers in your Exchange run this:

$TrackingLogResults = get-transportserver | where {$_.Name -like "<optional filter>"} | foreach-object  {Get-MessageTrackingLog -EventId DELIVER -MessageID <MessageID from Step1> -ResultSize Unlimited -server $_}

• Get-TransportServer gives you all the transport servers in the organization
• Where clause filters the servers list, you can leave it out, it is helpful if your HUB transport servers are named in a specific way, and you know the message did not leave the organization, so you can exclude a search on the Edge Servers.
• Foreach-Object cycles through all servers and performs the search
• Get-MessageTrackingLog searches each transport server tracking log for DELIVER Events that correspond to messages with that specific MessageID. It returns unlimited results. The server that is being searched is piped from the Foreach cmdlet.
• If you run the last cmdlet without the EventID filter, you will get lots of other EventID’s like fail,send,receive,routing,expand. You just need deliver, DELIVER is important because it basically says “OK, this message passed all of my checks I am now sending it to the Mailbox Server so it can submit it to the mailbox store”, so you get a list of just the actually affected mailboxes.

This may take a while to run. Once it is finished we have to get the list of people that the message was sent to. The easy answer would be “why not just do $TrackingLogsResults | select-object Recipients and pipe it along to something else?”

Well you can do that, but in some cases Recipients means actually a bunch of other addresses, and each recipient may appear multiple times in the entire list.

e.g. – this could be a list returned by the “easy” command

{john@foo.com}

{John@foo.com,Jane@foo.com}

{Jill@foo.com,Josh@foo.com,Jake@foo.com}

Having duplicates is inefficient, everything will take longer in next steps. What I wanted was to have a list without duplicates, plus I get to show you some more “nice” scripting stuff

Compile Recipients List

I spent quite some time figuring this out, so someone out there better find it useful . The next step involved a “google shovel” to “dig up” how to break up those objects into one big list. Then the plan was to have a list that just had the unique email addresses – ideally. So here’s the “magic”:

$RecipientsExpanded = @()
$RecipientsExpanded = $TrackingLogResults | foreach-object {$RecipientsExpanded  = $RecipientsExpanded  + ($_.Recipients)}
$RecipientsGrouped = $RecipientsExpanded | group-object
$UniqueRecipients = $RecipientsGrouped | select-object Name | sort-object -property name

• We created a blank array object that will host all recipients addresses in “expanded form”.
• For each result from the TrackingLog we added the array ($_.Recipients) to the $RecipientsExpanded array. At the end of this we have a single array with all the addresses, each an individual element in the array.
• The Group-Object cmdlet is used to group all addresses by their name and in the end you have the list of unique email addresses.

Actually remove offending messages

Please see this link if you are planning to export the messages to PST. What is left to do is to take a page from the MSExchangeTeam blog and run get-mailbox| export-mailbox combo, only we are doing it on a reduced scale, only on the mailboxes that need it, that why I went through all the trouble of making that list!

$MailboxesList = $UniqueRecipients | foreach-object {
      $Filter = "PrimarySmtpAddress -eq '"+$_.Name+"'"
      get-mailbox -ignoredefaultscope -resultsize unlimited -Filter $Filter}

The code above handles this task for forests with child domains. I covered reasoning and use of -Ignoredefaultscope and -Filter in a previous post.

#get current admin UserPrincipalName
$Admin = [Security.Principal.WindowsIdentity]::GetCurrent().Name
#elevating the administrator's account to fulll access over all affected mailboxes
Add-MailboxPermission $MailboxesList -AccessRights FullAccess -User $Admin
export-Mailbox -Identity $MailboxesList –ContentKeywords <enter part of message body> -Recipients <add recipients list> –TargetMailbox admin_ –TargetFolder "RecoveredEmails" –DeleteContent

• The final step grants the admin user full access over the mailbox. The account being granted that right is $Admin, the account under which the script is running, it contains the UserPrincipalName of that account.
• You also need to have admin rights on the “TargetMailbox” and the “TargetFolder” should also exist beforehand.
• We export the offending message(s) using Export-Mailbox. Here it is important to be very careful and make the filtering as strict as possible, since here you cannot remove a message based on the MessageID, so you could end up removing many more messages. Refer to the documentation for export-mailbox, for all available switches for this purpose.

After you run the last command get ready for some really long waiting, as it goes through all the mailboxes. Once it is finished, remove your permissions from those mailboxes.

Remove-MailboxPermission $MailboxesList -AccessRights FullAccess -User $Admin

Phew this was a long post, but validating everything I explained here, took a while. The post is also packed with bits and pieces that can be your building block for other Exchange Shell scripts. I tried to show you how to take Exchange TrackingLog data and build a list of unique recipient addresses that you can use to filter out an unwanted message you tracked in the logs, and do that using export-mailbox commandlet. If you have any feedback/corrections/omissions please feel free to leave a comment.

Happy Scripting!

Task #1 - Get a list of all legacy mailboxes that have forwarding addresses enabled or that DO NOT have a forwarding address at all.

and then

Task #2 – Based on the list from 1, create a report with forwarded email address and the forwarding address.

Task #1 – Is done via this one liner

$LegacyFWList = get-mailbox -ResultSize unlimited -RecipientTypeDetails LegacyMailbox -IgnoreDefaultScope -Filter {(ForwardingAddress -like '*')} | select-object ForwardingAddress,WindowsEmailAddress,RealFWAddress

Code Breakdown:

get-mailbox -ResultSize unlimited -RecipientTypeDetails LegacyMailbox -IgnoreDefaultScope -Filter {(ForwardingAddress -like '*')} -ResultSize unlimited

• By default the cmdlet returns only the first 1000 results. I used unlimited but it also accepts a number as input

• get-mailbox cmdlet extracts only “LegacyMailbox” objects – this means mailboxes on servers pre Exchange 2007.

• IgnoreDefaultScope performs a forest-wide search. Without this only legacy MBX’s on exchange servers in the forest root domain would have been returned.
• Using this does have some limitations to it, check the help for what those are.
• Biggest limitation is that -Identity must now be the a valid DN of the object.
• I did not want to extract all DN’s in the forest and pipe them to the cmdlet, I looked for an alternative to this.

I discovered the –Filter parameter. It allows you to apply filters to the search on more Object Properties, much better than –Identity which we cannot use because of the -IgnoreDefaultScope.

A great thing about it is that it can be used without any limitations with IgnoreDefaultScope. Full details about how to use –Filter Parameter and what properties of AD objects are filterable can be found here and here. Back to our task, you can see what we did.

-Filter {(ForwardingAddress -like '*')}

We looked for any objects that contained any value in the “ForwardingAddress” Property of the returned object

But say you want the opposite of this “get all legacy mailboxes that have no forwarding address”…if would stand to reason that this should work:

-Filter {(ForwardingAddress -notlike '*')}

You are out of luck, it returns the entire list of legacy recipients. I went through a few filters until I found this:

-Filter {(ForwardingAddress -ne $null)}

or

-Filter {(ForwardingAddress -ne*')}

I have to say that it was a surprise when I stumbled upon this, as it doesn’t really make sense. TechNet articles say -like, and I assume also -notlike accept wildcards, while -eq does not, you can also assume -ne does not. Well -ne does take wildcards, which I particularly do not mind in this case. I’m not sure if this is a bug, a feature, or just a flaw in my logic somehow and it’s working because it is supposed to.

Moving on…

select-object ForwardingAddress,WindowsEmailAddress,RealFWAddress

• I can save you running a “| get-member” on what the get-mailbox returned previously and tell you that only “ForwardingAddress,WindowsEmailAddress” are properties returned by get-mailbox.
• “RealFWAddress” is something I made up. Why? Well unless you know already ForwardingAddress is not an address actually, it is a collection of properties for the recipient where emails are forwarded, the list contains DN, ObjectGUID, CN and others, not an actual email address.
• RealFWAddress is a blank property, attached to the $LegacyFWList object, as a placeholder for when we do script a way to get that email address…which is now! read on

Task #2 – Create a list of forwarded addresses and forwarding addresses.

Now we come to the second part of my post, the “get-recipient” cmdlet. Since there is no way for us to know to what object are the emails forwarded to [it could be a DL, a contact, a mailbox, another legacy user] we have to make our search as broad as possible. If you try to run get-mailbox on a contact object you will get nothing obviously. We are using foreach-object to go through the entire list. ForwardingAddress is actually the CN of the object. We pass the DN of the Property to a cmdlet that will return a primary email address, if any.

This following code will accomplish just that:

$LegacyFWList | foreach-object {
 $ForwardingDN = $_.ForwardingAddress.DistinguishedName
 $Filter = "DistinguishedName -like '$ForwardingDN'"
 $_.RealFWAddress = get-recipient -Filter $Filter -IgnoreDefaultScope | select-object PrimarySmtpAddress}
$LegacyFWList | export-csv "c:\Change\This\Path.csv"

Code Breakdown:

$ForwardingDN = $_.ForwardingAddress.DistinguishedName
$Filter = "DistinguishedName -like '$ForwardingDN'"

• I used these 2 variables in the code, $DistinguishedName and $Filter, trying to write a one liner resulted in somehow “deformed” filter command that get-recipient would not understand.

$_.RealFWAddress = get-recipient -Filter $Filter -IgnoreDefaultScope | select-object PrimarySmtpAddress

• Lastly, RealFWAddress property I introduced above gets populated with PrimarySmtpAddress value.

$LegacyFWList | export-csv "c:\Change\This\Path.csv"

• In the end all that’s left is export the $LegacyFWList variable to CSV and process however you wish, or just store it in xml / pass it on to another part of a script.

I have to admit the last part of the code is less ideal. Why? Well, the export does not output a clean forwarding email address. I have tried various ways to fix it, but in the end I concluded that it took me 30 minutes of trying to get a nicer export, and 15 seconds to fix it after importing the csv in excel.

UPDATE 1: 26.01.2010 – added proper way to filter people with no forwarding address
UPDATE 1: 21.02.2010 – added syntax highlighting and small rewrites

Powershell is Microsoft’s new (or maybe not so new, let’s just say latest , it has been with use for some years) extensible command shell interface. It is built on .NET Framework, so it may look more like a shell aimed at the programming savvy, but I think people that have really basic programming skills can handle it after some getting used to.

How to get it? Well assuming you have an Exchange 2007 server in your organization, you have it by default on the Exchange Servers. You can and should use the shell on the Exchange Servers [running scripts closest to the "managed entities" is always best IMO, you avoid adding additional Points of Failure] but you can also download the Exchange Management Tools for Exchange 2007 and install them on your own workstation, or a workstation you designate for management.

Ok, so now you have the tools installed. Unless you already found the link to the Exchange Management Shell on your start menu, you can start the tools by typing this in the run box.

%windir%\system32\windowspowershell\v1.0\powershell.exe -PSConsoleFile "%ProgramFiles%\Microsoft\Exchange Server\bin\exshell.psc1" -noexit -command ". '%ProgramFiles%\Microsoft\Exchange Server\bin\Exchange.ps1'"

You can probably use this in some scripted NT batch files, if you like. You will find some good information on Google about introductions to powershell, I won’t bore you with them. I’m going to assume you have the shell, have done some really light reading on the topic.

Here are some quick fixes for Powershell’s quirks though. You must have stumbled upon this at least once.

Ever ran an extended help command like the one below?

get-help Get-MailBox -full

Lots of lines scrolling, you couldn’t read a thing. It’s like one of those days you wish you were either in The Matrix or Startrek, so you can read the damn command-let help slowly or be an Android…or you could have not skipped the RTFM part of powershell, but if you’re like me, you did.

Worst part about this is that the powershell.exe line buffer is the same as the buffer of cmd.exe, so if the help is big enough, scrolling back up won’t give you a change to read all of the help.

How to Fix it? – Not to worry old NT batch and Powershell have the answer

Tip #1 – Using Powershell - For this I should have done the obvious, meaning taking a closer look at the output of:

get-help get-help (yes you can get-help on get-help)

There’s a little line that says there’s also the option of running, help [command] instead of get-help. You you try:

help get-mailbox -full

You get the equivalent of the /p switch on the NT batch, now that’s help for non Android people

Tip #2 – Oldschool, using NT batch you can redirect the output of help to a text file and read it at your leisure.

get-help Get-mailbox-full > c:\get-mailbox.help
notepad C:\get-mailbox.help

Which one is better?

Powershell method is quicker because you don’t have to notepad the text file and search for it there. However you cannot use the same method for cmd-let output [or may I have not figured out how, yet], basically reading the output of a cmd-let page by page.

For that you still have to do use the general method, in NT Batch redirect the output to a file and read that later on, like this for example:

get-excommand | select-object Type,Name | ft > c:\ex-command.help

notepad C:\ex-command.help

This should help anyone struggling with learning the Exchange shell syntax for referencing all of the cmd-let switches available.

Happy Scripting, hope this first post was not so bad.