Ionuț Nica
Win, VM, Linux, Scripting – ramblings of a SysAdmin
Restoring mailboxes in Exchange 2007 (part 1)
Jul 15th
Lately I’ve been doing a number of mailbox restore procedures on Exchange 2007, so I thought it would be a good idea to make my own posts about it (yes it involves scripting), because things are not always as straightforward as MS or TechNet say it is. This is going to be a multi-part post: Create the RSG and mount the DB to be restored, Restore mailbox(es), Remove the restored DB and RSG. Before you think about it I’m going to answer it for you:
Q: But why don’t we use the nice GUI Tool from Exchange Management Console (Extra.exe) and do it from there, “we don’t need no scripting”?
A: My experience tells me the scripted method is safer and works “as expected” unlike the GUI, which says it did something, when it didn’t (I’ve spent days trying to figure out why a RSG Database won’t actually dismount when the GUI said: “Completed Successfully”.
OK, let’s get on with it. All that I am about to explain requires Exchange Administrator privileges on the Exchange servers.
We will be creating a Recovery Storage Group, this is the first step in the restore process. To create the RSG you need following:
- Adequate disk-space to restore the mailbox database, locally on the Exchange Server where the DB was residing
- Exchange Management Shell running as Administrator (especially on CCR clusters)
- No other Recovery Storage Group already created on that server with an existing RSG database (you can only have 1 RSG with 1 DB in the RSG). It is best to remove any previous RSG completely then recreate it for your needs.
- Specific information like which DB to link to the RSG and the list of mailboxes to restore.
Creating a Recovery Storage Group can be as easy as this:
New-StorageGroup -Server <MBX Role Server Name> -Name <StorageGroup Name> -LogFolderPath <Logs Folder> -SystemFolderPath <SystemFiles Path> -Recovery -Verbose
The command is very similar to creating a new SG, except for the -Recovery switch, designating it as a Recovery Storage Group. I added the -Verbose switch so you can see what is going on behind the scenes.
New-MailboxDatabase -MailboxDatabaseToRecover <Mailbox Name> -StorageGroup <Recovery Storage Group Name> -EdbFilePath <path to store edb file> -Verbose
Here it is just as easy as creating a new mailbox database, only you are creating it in the recovery storage group you created with the previous command. The key thing to remember here is that the value of the “MailboxDatabaseToRecoverParameter” must be the exact same name of the mailboxDB of which you want to recover from. If the name is different you will not be able to run any restore commands, because it will not be able to find any mailboxes when it searches the recovered database.
A working script for creating the RSG
Below I’m sharing with you a working snippet that should help in creating a recovery storage group and DB. In short here is what the code does:
Using a given UserPrincipalName…
- Attempts to retrieve the mailbox for the UPN (it is a “forest friendly” coding for retrieving the mailbox). If it fails it quits
- Checks if a folder structure for placing, logs, system files and the edb file exists (I used a location called d:, use a variable if you like).
- If folders already exist, it will quit, otherwise it will create a folder with the MDB name, and logs and edb subfolders,
- Next it checks if a Recovery Storage Group already exists, unless you cancel the script it will continue to use this RSG, with the given details. Otherwise it will create a RSG on its own.
- It will then create a mailbox database where you / your backup admin will restore your exchange backup.
$MBX_UPN = Read-Host
$Filter = "UserPrincipalName -like '$MBX_UPN'"
$SourceMBX = get-mailbox -IgnoreDefaultScope -Filter $Filter
If ($SourceMBX -eq $null) {
Write-Host -foregroundcolor Red "No Mailbox for $MBX_ID found`nScript will Quit"
exit }
Write-Host -ForegroundColor Green "Source Mailbox is`n $SourceMBX"
$LinkedMDB = Get-MailboxDatabase -Identity $SourceMBX.Database
Write-Host -ForegroundColor Green "Ok, Database ($($LinkedMDB.StorageGroup.Name)) is grabbed, now creating RSG Folders and RSG`nPress Enter to continue or Ctrl+C to Cancel"
Read-Host
#Checking if the RSG folders already exist, if not attempt to create them
If ((Test-Path "d:\$($LinkedMDB.StorageGroup.Name)")) {
Write-Host -ForegroundColor Red "Folder already exists. Please remove d:\$($LinkedMDB.StorageGroup.Name) before running this script again.`nScript will quit"
exit
}
$SysPath = New-Item -Type Directory -Path d: -Name $LinkedMDB.StorageGroup.Name | Get-Item
If ((Test-path $SysPath)) {
$DBPath = New-Item -Type Directory -Path $SysPath -Name DB | Get-Item
$LogsPath = New-Item -Type Directory -Path $SysPath -Name Logs | Get-Item
}
#If folders were created successfully we can continue
If ((Test-path $SysPath) -and (Test-path $DBPath) -and (Test-path $LogsPath)) {
#Checking if RSG already exists
$RSG_check = Get-StorageGroup -Server $LinkedMDB.ServerName | where {$_.Recovery -like "True"}
If ($RSG_check -ne $null) {
Write-Host -ForegroundColor Magenta "A RSG was found on $($RSG_check.ServerName). Here are RSG Details:"
$RSG_Check | select-object Name,Identity,Recovery,LogFolderPath,SystemFolderPath | fl
Write-Host -ForegroundColor Magenta "To use this RSG Press Enter, to cancel Press Ctrl+C"
Read-Host
}
Else {
Write-Host -ForegroundColor Green "Now creating Recovery Storage Group..."
New-StorageGroup -Server $LinkedMDB.Server -Name "Recovery Storage Group" -LogFolderPath $LogsPath.FullName -SystemFolderPath $SysPath.FullName -Recovery -Verbose
}
Write-Host -ForegroundColor Green "OK! No RSG found. Now creating RSG Database..."
New-MailboxDatabase -MailboxDatabaseToRecover $LinkedMDB.AdminDisplayName -StorageGroup "$($LinkedMDB.ServerName)\Recovery Storage Group" -EdbFilePath "$($DBPath.FullName)\$($LinkedMDB.Name).edb" -Verbose
}
Else {
Write-Host -ForegroundColor Red "Could not Create folder or folder structure in d:\$($LinkedMDB.StorageGroup.Name). Check messages above for errors! Script will quit."
exit
}
This is about it with creating a Recovery Storage Group, it is actually not difficult, just remember to name the MDB inside the RSG with the same name as the source MDB (this was also required on Exchange 2003, as far as I know). Also you cannot have more than one RSG per Maibox Server, it is best to remove any RSG you have after you are finished recovering data. Next post we will discuss how to restore data from a MDB and how to remove the RSG.
As always I value your feedback and hope you found this post useful.
Fix High Hardware Interrupts on HP Probook 6540b
May 27th
A few days back I discovered an “issue” with new installation of Windows XP on HP Probook 6540b laptops (have read reports on the internet about 6440b behaving the same). What I noticed is that after installing all the drivers from the HP website, computer appeared sluggish, especially disk operations, opening task manager was a 5 second task. The solution to this problem was to obtain the latest disk controller drivers from the Intel Website and then install a specific controller type, instead of letting Windows choose automatically. But first, time for this small disclaimer:
The steps below should be attempted after you have backed up your Windows installation and/or relevant documents, please do not attempt this procedure before doing a backup of your system. Double, triple, quadruple check that the problems I am describing here exactly match your hardware, software and symptoms observed. This procedure can damage your operating system, possibly even the hardware, this post comes with no warranties, it is not supported by HP, Intel or any other vendor as far as I know. Also this post is valid at the time of writing, new fixed drivers may appears by the time you are reading this, making it obsolete.
Symptoms and Conditions
- Sluggish Disk performance
- Sluggish computer performance when doing disk based operations
- BIOS is configured to use IDE mode not AHCI mode for SATA disks (you configured for IDE because AHCI was not working)
- Device Manager is showing a primary IDE channel device configured for PIO mode only – you cannot select UDMA mode
- Using SysInternals Process Explorer reveals 25% CPU is Hardware Interrupts when accessing disk continuously (on an i5 cpu that is 1 core…spread over all 4 cores). Interrupts usage goes down when disk is idle.
- Your storage controllers are detected as:
- “Intel(R) 5 Series/3400 Series Chipset Family 2 port Serial ATA Storage Controller”
- “Intel(R) 5 Series/3400 Series Chipset Family 4 port Serial ATA Storage Controller”
- The Storage Controllers detected above have Hardware ID’s:
- PCI\VEN_8086&DEV_3B2D
- PCI\VEN_8086&DEV_3B2E
Trials and Errors
As you can see starting from the High Hardware interrupts up to device being put into PIO mode, it is clear that there is some driver issue somewhere. What you can try and watch it fail:
- Delete disk controller devices then use scan for hardware changes so windows will reinstall drivers. After reboot you will see the problem still exists.
- Delete disk controller devices then download latest Intel drivers, automatically choose which driver to install. After reboot you will see no changes, same device will be detected, same drivers installed.
- Reinstall OS re-add drivers one by one, you will have the same problem.
The solution
You need to manually select a device driver to install from the driver package for Intel’s Rapid Storage Technology (I think the former name for these drivers, in general, was Matrix Storage Manager). The problem is that the drivers that come with Windows can only use PIO Mode, and the driver package from Intel does not contain the Hardware ID’s you found above. The drivers do work, to get them to work you have to either:
- Manually install drivers selecting a specific device driver to install
- Hack the driver files so they include your device ID’s (“hardcore” option, try #1 before you go there
)
Option 1 – Do a manual driver install
This works mostly after you installed the operating system. here’s what you need to do exactly:
- Download the driver package
- Using Device Manager delete storage controllers from the laptop
- In Device Manager click “Scan for hardware changes“, the New Hardware Wizard appears
- Choose don’t search Windows Update
- Next choose “Install from specific location“
- Next choose “Don’t search I will choose driver to install“
- On the next screen click on “have disk” and point it to the location of the extracted driver files. Click OK to close driver selection. List will be populated with a bunch of devices
- From the devices list select “Intel(R) 5 Series 6 Port SATA AHCI Controller“
- Click OK and correct drivers should install now. If you are asked to reboot, choose OK
- After the reboot go into the BIOS, change SATA mode to AHCI. If you keep SATA mode to IDE your XP install will BSOD (the reason is you added SATA drivers to XP, and the controller talks IDE, if left unconfigured)
- Now you should see that your devices are installed correctly and you have no more hardware interrupts. Also the Disk Controllers section in Device Manager looks different, fewer devices left there.
Option 2 – Hack the Driver files
This option is useful if you want to make a driver package for an unattended installation or just want to have a set of drivers that will work “out of the box”. What we will do in short is add a few lines of code to the files in the driver package, pointing the Hardware ID’s to the “Intel(R) 5 Series 6 Port SATA AHCI Controller” we manually installed with Option 1. Here’s how to do it:
- Extract drivers to a folder, you should have these files inside among some other txt’s:
- iaAHCI.cat,iaAHCI.inf,iaStor.cat,iaStor.inf,IaStor.sys,TXTSETUP.OEM
- Open iaAHCI.inf file for editing and search at the end of the file for the “strings” section. Look for the string “PCI\VEN_8086&DEV_3B2F&CC_0106.DeviceDesc” which matches to the Intel 5 series 6 port controller . As you can see after the DEV_ follows “3B2F”, pretty similar to our Hardware ID’s:
- PCI\VEN_8086&DEV_3B2D
- PCI\VEN_8086&DEV_3B2E
- Before the DEV_3B2F line create 2 new lines where you duplicate the DEV_3B2F line, BUT you replace 3B2F with the last 4 characters from the other device ID’s (one line will have 3B2D the other 3B2E). The point is to have the Hardware ID’s of your controller point to the correct driver name.
- Now we have to track any place in the document where “3B2F” appears and add the same text for Hardware ID’s 3B2F and 3B2E. The section you are looking for to add lines are in “[INTEL_HDC.ntx86]“, there is a line containing 3b2f, add 1 line for each Hardware ID.
- Save iaAHCI.inf and close it
- Update Disk Controller drivers by pointing Windows hardware wizard to your modifed .inf file
With this inf file Windows should be able to install the driver it needs without you having to select which driver to install from the list. The logic is that now Windows knows where to find the correct drivers, because the Hacked Intel Driver contains the device ID’s Windows is looking for.
Option 2+, unattended installs
This next section is about changing the TXTSETUP.OEM so you can do unattended installations using this hacked INF file. You can follow the Intel guide to injecting drivers for “F6 Install”, but you need to change the TXTSETUP.OEM file that comes with this package. Do following:
- From the driver package Open TXTSETUP.OEM for editing.
- In the iaAHCI.inf section look for the “Intel(R) 5 Series 6 Port SATA AHCI Controller”. To the left of that string is the text “iaAHCI_5_1″.
- Do a search for the string “iaAHCI_5_1″ in the document, you should find a section called “[HardwareIds.scsi.iaAHCI_5_1]“.
- When found copy it and the line after it ( looks like “id = “PCI\VEN_8086&DEV_3B2F&CC_0106″,”iaStor”") 2 times. The 2 copies you can change instead of being 3B2F to 3B2D and 3B2E respectively.
- In the end you should have 3 “hardwareIDs” sections, 1 with 3B2F, the original and the other 2 Hardware ID’s you need.
- Save and close TXTSETUP.OEM.
- Follow Intel’s “F6 install” procedure to deploy Windows XP using these modified files (all the files in the package + modded iaAHCI.inf and TXTSETUP.OEM)
- You must configure BIOS to use AHCI mode, drives will not work with IDE mode (didn’t for me)
Phew, this was a long and “hard” post. I hope the general idea is clear:
For installing from windows just make sure to select the controller I mentioned (the 6 port device) when doing the complete manual install.
For the hacked inf and OEM files double check and triple check the changes you are making. the point is to add the HW ID’s to the INF file, so it will install the drivers the same way as for the Hardware ID ending with 3B2F.
My best guess is that this mess-up is due to some slightly different versions or ID’s being stamped erroneous onto the controllers when they were shipped. I hope this was helpful, please report back any mistakes you notice.
Log Battery and Power Levels using Powershell
May 9th
This is a let’s say lighter post, I came up while trying to compare battery life of my laptop and some buddies of mine. I wanted to know, how fast my battery depleted using different settings, use profile and power saving modes. Then I did some digging around Microsoft’s MSDN site, and I found some interesting WMI classes, that apparently provide a lot of “power related data”. I also wanted to have a way to log this data, and that’s how I ended up learning how to create a new event-log file and write data to it to use that as a log. So this is what I will try to show: get power related data and write it to the Event-Log.
“Energy” Related WMI Classes
Here are a few interesting classes I stumbled upon. Some of them are only available under Windows7 probably also Vista, but I’m not sure.
- WmiMonitorBrightness – gives information about monitor brightness. For example these line give the max. “value” and current value of brightness
$MaxBrightness = get-wmiobject -class WmiMonitorBrightness -Namespace root/wmi).level | measure-object -Maximum).maximum
$CrtBrightness = "{0:P0}" -f ((get-wmiobject -class WmiMonitorBrightness -Namespace root/wmi).CurrentBrightness/$MaxBrightness)
- Win32_PowerPlan – provides information and identifiers about the powerplans defined. In this class ALL powerplans are defined, and just the active plan has an “IsActive” flag attached it, here’s how to get it:
$powerplan = (Get-WmiObject -Class win32_powerplan -Namespace 'root/cimv2/power' | where {$_.IsActive -eq $true}).ElementName
- Win32_Processor – gets information about the CPU (I was interested in the CPU load for statistical purposes). This one was pretty easy to find, the value was written in plain sight. Take a look:
$cpu = (Get-WmiObject Win32_Processor).LoadPercentage
- Win32_Battery – Provides information about the battery itself (estimated time, remaining load, power status). Running “Get-WmiObject -Class Win32_Battery | gm” take a closer look at these members:
- BatteryStatus – this will toggle between ’1′ meaning on Battery and ’2′ meaning on AC Power
- EstimatedRuntime – this will be the number of minutes running on battery, as the OS estimates it, and if you get a very high value (tens of thousands) when you plug the AC Power, it means the battery is charging
- EstimatedChargeRemaining – percentage-wise representation of battery charge remaining
Powershell + Event-Log “101″
I used this battery and power experiment to learn more about working and writing data to the Event-Log. I wanted to create a new “Event-Log” in Windows (windows 7 as you probably know allows for a lot of application logs) and then write events to it. Then at any point you can export the Event-log to csv. The following creates an Event-Log, with the name “BatteryMonitor” from the Information category (for my uses “Source” was not needed but it is a required parameter:
New-EventLog -Source BattMon -LogName BatteryMonitor -CategoryResourceFile Information
You can also check if an Event-log is created exists you can use this scriptlet (the answer lies in WMI this time, I didn’t find a cmdlet that does it faster):
(get-wmiobject -class "Win32_NTEventlogFile" | where {$_.LogFileName -like 'BatteryMonitor'} | measure-object ).count -eq '1'
Finally here’s how to write to the event-log, a new event. This bit I used in a script to mark the execution of the script in the event-log:
Write-EventLog -LogName BatteryMonitor -Source BattMon -EventID 65533 -Message 'Starting new Execution of BatteryCharge Monitor Script. The script will pump here CSV values. Values are listed in this order, as CSV: PowerPlan,PowStatMsg,ChargeRemMsg,RemTimeMsg,RAM,CPU,CrtBrightness' -EntryType Information -ComputerName $env:computername -ErrorAction:SilentlyContinue
So that is about it, as usual I tried to tie all of these scriptlets into a usable script, you can download it from here.
Fix “Transaction log for database ‘VIM_VCDB’ is full” errors
Apr 19th
This is one of those “note to self posts”, in hope this may hit me again so I don’t go wandering the Internet all over again. I have a small VMware lab at home, and a few days ago I was confronted with an issue related to vCenter – the management application for VMware’s hypervisor. I tried to connect to my vCenter installation – connection refused….ok, I’ve seen this before, probably the service is not up. Initially I thought there had been a power outage at my home (they kinda happen) and the vCenter Service hanged upon starting (this also kinda happens)
No problem I can fix it! open services snap-in remote to vCenter machine, start service, service starts, close snapin. Start vSphere Client client works, play around with it a bit, close Client.
Time goes by, I need to log back into the system again for some work. Connection refused….now this is rich, no power outage, why is the service crashing? Ok, it’s just life treating me badly VMware is acting up (not that is usually does), open service, start service, login again to vCenter, do some work, few minutes later client disconnects…reconnect not working.
Ok, troubleshooting mode now; open Splunk, sort by events from that host, anything that is not information from the system log. And there it was:
Error[VdbODBCError] (-1) “ODBC error: (42000) – [Microsoft][SQL Native Client][SQL Server]The transaction log for database ‘VIM_VCDB’ is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases” is returned when executing SQL statement “UPDATE VPX_VM WITH (ROWLOCK) SET SUSPEND_TIME = ? , BOOT_TIME = ? , SUSPEND_INTERVAL = ? , QUESTION_INFO = ? , MEMORY_OVERHEAD = ? , TOOLS_MOUNTED = ? , MKS_CONNECTIONS = ? , FAULT_TOLERANCE_STATE = ? , RECORD_REPLAY_STATE = ? WHERE ID = ?”
Ouch, something really broke, Immediately I made quick check to see if I had disk space left, which I had, so this was not going to be this easy.
In that case: to the Internets! Found this thread on the VMware communities. I won’t bore you anymore with the storyline, I’ll just get to fixing this issue
Note: this is probably an extremely trivial topic that does not happen on production databases, with vigilant DBA;s. However this is a homelab and I’m not a DBA and if you are reading this, probably so are you.
The Fix
To fix this you will need SQL Server Management Studio Express installed either on the server holding the databases or on a management machine (in which case you better know how to give yourself remote access to the vCenter Database Server, I couldn’t, so I installed it locally on the affected machine). You’l also need a local administrator account to run the management studio under.
Once in the management studio, select the VIM_VCDB database, right click properties:
On the left side of the new window select the File section:
So, there are 2 files, database and the logs. The error we got mentioned log files. A quick look in my setup revealed I had reserved only 460MB for logs (screenshot taken after fix). Scroll down to the right, and find the “…” button, which will let you configure the maximum size of the log files. 
Now change this value to a bigger value, for a home lab 2GB is quite a lot actually, but i wanted to be safe. Close all windows by pressing OK, close the Management Studio.
After this restart VMware VirtualCenter Server service and watch your vCenter go .
Now for a little investigation why this happened. The vCenter database holds performance data, VM metadata and the likes…but how could 8VM’s gather performance data in less than 2 months that fit into 460MB which was the configured size of the log file….Well the answer lies into vCenter Server Settings, once I started browsing the menus I remembered, that just for testing I configured the statistics logging level to 4 (highest) for each retention period, and not just for testing, I Forgot to turn it off, lesson learned now.
p.s. This my first non scripting post
Restrict USB Storage Devices on Windows XP
Apr 9th
This is one of those topics that are probably on the top 10 to do’s of anyone’s list when it comes to securing their Windows desktops. Whether it is plain dictatorship, security/confidentiality concerns/requirements, unpatched OS’s, weak/no AV solutions, the golden POLP (Principle Of Least Privileges) may force you to come up with a solution to this problem. If you are using anything else (XP, 2000, 2003 Server) except the newer versions of Windows (Vista, 7) which allow you to do this via a GPO setting, you are out of luck, there is no GPO setting or quick-fix that works.
As a short history, I went through CIA documents that were published (can’t find them anymore), Forums, Microsoft KB’s, Whitepapers, and finally came up at the other end of the tunnel with a working process.
The goal is to devise a process of denying access to USB Storage that meets following criteria:
- Must be implemented at OS level
- Must be deployed scripted/automatically and/or via GPO
- Must not cripple other OS functionality (e.g. installing printer/scanner drivers)
- Must be fully reversible by Administrators only
- Must be working regardless if USB Storage was used before the process is put in place
The solution – explained
For disabling USB Storage there are 2 situations to cover:
- No USB storage ever installed, user must not be able to install device
- USB storage was previously installed by user or admin, user must not be able to use USB Storage again
Both scenarios are covered in these 6 steps:
- Copy usbstor.inf, usbstor.pnf, usbstor.sys to their default locations, as if a USB storage device would be installed.
- Restrict access to the 3 files mentioned above. We will use an implicit DENY for the local “SYSTEM” Account for these files.
- Remove Registry Keys that handle USB Storage device startup: HKLM\SYSTEM\CurrentControlSet\Services\USBSTOR and HKLM\SYSTEM\ControlSet001\Services\USBSTOR and HKLM\SYSTEM\ControlSet002\Services\USBSTOR
- Replace USB Storage related registry keys with specially crafted keys that disable startup of the USB Storage driver
- Apply an implicit DENY for the local SYSTEM Account on the Registry Keys mentioned above
- Insert USB Storage device, wait for it to be detected by OS and marvel at the fact it won’t let you install the device
For enabling USB Storage these steps must be taken from an Account that is member of the Administrators Group
- Remove restrictions placed on the ubstor.* files.
- Remove following specially crafted Registry Keys: HKLM\SYSTEM\CurrentControlSet\Services\USBSTOR and HKLM\SYSTEM\ControlSet001\Services\USBSTOR and HKLM\SYSTEM\ControlSet002\Services\USBSTOR
- Remove restrictions placed on the registry keys from above
- Delete incompletely installed USB storage devices fron Device Manager and Reboot Computer
- Insert USB Storage device, wait for it to be detected by OS/go to device manager and refresh device list and marvel at the fact it works
Implementation – explained
For implementing this in a scripted manner we will use batch scripting, I’m going for a low level approach, assuming you don’t have vbs / powershell on hand, vbs would be rather complicated anyway and Powershell is not installed by default on the OS. You do have some prerequisites:
- reg.exe (available by default on XP)
- A network share
- set-acl (open source utility – get it, copy to a network share of choice and be happy it exists)
Disabling USB
- The 3 usbstor files mentioned earlier, 2 are available by default (usbstor.inf and usbstor.pnf) under %WINDIR%\inf. The 3rd, usbstor.sys, unless a usb storage device was previously installed is not present. Find it under %WINDIR%\Driver Cache\i386\Sp3.cab or the other cab files there. Extract it from the cab file to the network share.
- The piece of code that disables USB is written below, but requires that set-acl, the specified .txt, .reg, usbstor.sys files be present in the same directory from which it is executed
::Copy ubstor.sys xcopy /R /H /Y %CD%\usbstor.sys %windir%\system32\drivers ::Secure USBSTOR.* files with ACE (only Local Administrators Full Control, local "SYSTEM" denied Full Control) SetACL.exe -on "c:\windows" -ot file -actn restore -bckp "%CD%\usbstor_ACL.txt" ::Delete settings related to USBSTOR Service REG DELETE HKLM\SYSTEM\CurrentControlSet\Services\USBSTOR /f REG DELETE HKLM\SYSTEM\ControlSet001\Services\USBSTOR /f REG DELETE HKLM\SYSTEM\ControlSet002\Services\USBSTOR /f ::Add special crafted registry keys regedit /s "%CD%\disable_usb.reg" ::Secure keys from above with ACE (only Local Administrators Full Control, local "SYSTEM" denied Full Control) SetACL.exe -on "hklm\SYSTEM" -ot reg -actn restore -bckp "%CD%\HKLM_ControlSet.txt"
- Line 5 of the code uses a file that contains a specially formatted ACL applicable to the 3 usbstor files. To generate a different ACL, use the syntax below for each file you are interested in. When you are finished you can merge all text files in a single text file and add it to the script.
SetACL.exe -on "c:\windows\inf\usbstor.inf" -ot file -actn list -lst "f:sddl;w:d,s,o,g;s:b" -bckp "%CD%\usbstor_inf_ACL.txt"
- REG command is used to delete any data that may exist in the specified registry keys (think previous installed USB Storage)
- Once the Registry is clean of the keys, we then push a customized reg file (find it at the end of the post), that essentially changes this:
USBSTOR driver points to the file you defined (usbstor.sys, that you just set a restrictive ACL on)
DeviceCount equals zero
DeviceStartUp Type is set to Disabled (more details here)
Other standard settings for that key
- Line 16 of code, similar to the ACL for USBSTOR Files, configures the security for the registry keys we added. To customize the ACL, change it to your liking then export the ACL using the command below and update the batch code to include it.
SetACL.exe -on "hklm\SYSTEM\CurrentControlSet\Services\usbstor" -ot reg -actn list -lst "f:sddl;w:d,s,o,g;s:b" -bckp "%CD%\HKLM_CurrentControlSet.txt"
Enabling USB
This is just a question of reversing the changes made by the Disabling process. The following piece of code does just that:
::enable inheritance of permissions SetACL.exe -on "c:\windows\inf\usbstor.inf" -ot file -actn setprot -op "DACL:np;SACL:np" SetACL.exe -on "c:\windows\inf\usbstor.pnf" -ot file -actn setprot -op "DACL:np;SACL:np" SetACL.exe -on "c:\windows\system32\drivers\usbstor.sys" -ot file -actn setprot -op "DACL:np;SACL:np" ::clear any non-inherited ACE SetACL.exe -on "c:\windows\inf\usbstor.inf" -ot file -actn clear -clr "dacl,sacl" SetACL.exe -on "c:\windows\inf\usbstor.pnf" -ot file -actn clear -clr "dacl,sacl" SetACL.exe -on "c:\windows\system32\drivers\usbstor.sys" -ot file -actn clear -clr "dacl,sacl" ::deleting custom Registry Keys REG DELETE HKLM\SYSTEM\CurrentControlSet\Services\USBSTOR /f REG DELETE HKLM\SYSTEM\ControlSet001\Services\USBSTOR /f REG DELETE HKLM\SYSTEM\ControlSet002\Services\USBSTOR /f
- As you can see we are enabling inheritance of permissions, clearing any ACE defined explicitly on that object (the ones we pushed actually) and removing the Registry keys we also pushed. Make sure the user running this enabling process has rights to change these objects (in our case he is member of the Local Administrators Group)
- After this is done manually clean it of any hidden installed USB Storage devices and reboot the computer. After the reboot replugging the device should allow you to install and use it again.
Phew, this was also a long post, but believe me, reaching this compressed format was a lot of work .
Now I’ve attached this zip file containing the contents of what I’ve been talking about, it should be usable out of the box.
There is also there question I guess of securing these files so that they apply to users but users can’t get to them to “help themselves”, but that is another topics for another post perhaps.
As always any feedback is welcomed.
Change Notification “From” Email Address in FSE
Mar 28th
After a recent deployment of FSE (Forefront Server for Exchange) on an Exchange CMS, I sat back and just watched Forefront notifications come in. Initially they were delivered in my Outlook junk email folder, but I quickly figured it out and added the “domain” Forefront was sending from. This was all ok for me, a sysadmin, but then it hit me: Users whose attachments get blocked also get this notification and it probably ends up in the junk folder aswell. First I thought, ok, let’s push a list of accepted domains down to the clients, but then it dawned on me that there had to be a way to change the “from” address in Forefront. As an added bonus I wanted to find a way to change it on any Server Role (mailbox, hub, cas, edge). Thankfully the Forefront Server for Exchange User Guide provides the answer (goes to show RTFM sometimes goes a long way).
How to Change the From Address
This From Address is written down in a registry key of the Server where FSE is installed.
- Open the Registry Editor and browse to the corresponding key depending on the OS version you have:
For 32-bit:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Forefront Server Security\ Notifications\FromAddress
For 64-bit:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Forefront Server Security\Notifications\FromAddress - Change the default value to the sender name you would like. I chose a name that also identifies the server where the notification is sent from. You can use any alphanumeric characters, just keep in mind that illegal characters are replaced with “_”.
- Now Microsoft says to Restart/Recycle the Exchange and FSE services for this change to become active, please read the notes below regarding this:
Notes: Restarting/Recycling Services looks like quite a simple task however, I do have some hints on that:
- If using a CMS (Clustered Mailbox Server) you just have to recycle FSE on the ACTIVE node, FSE is not actively running the PASSIVE node. Exchange services don’t need to be restarted for this change to take effect – (this is my personal observation)
- Recycling FSE on servers running Microsoft Exchange Transport Service (Transport, Edge Roles) will also recycle the Microsoft Exchange Transport service (Transport Service has a dependency on FSE) – so plan for downtime.
- There is a “bug” in Forefront: after the change and restarting services (not sure what is the cause, didn’t play around to find out) Forefront sends out 3 failed update notifications (1 per engine), for deprecated scan engines it has (AhnLab, Sophos, CAvet) even though none of those engines are enabled/configured to get updates. I spent almost 2 hours scouring the Internet for a reason why this happened. Best just to ignore it, save yourself some brain-cells.
- If changing the registry key by hand via remote registry you will not see the wow6432Node because regedit already connected you to that key.
Disclaimer: As you know changing the registry is “bad juju” if you don’t know what you are doing you can break things, so handle these operations with care.
This is probably the first post in a series related to Forefront Security for Exchange, I hope they prove useful.
Configure USB device to boot WindowsPE
Mar 21st
It’s been a while since my last post but now I found some time to write a complimentary article to my “make your PE boot disk series”. While the make your PE disk series showed you how to make a bootable ISO, it is possible however to write the contents of the ISO to a USB stick and by following this guide to boot Windows PE from USB. The advantage of having a WinPE on USB is great flexibility, with a CD you always have to rebuild the ISO and burn it/mount it whenever you feel like adding new applications/scripts to it. With a flash drive you can just copy the data alongside the booting OS.
Here’s what you need:
- USB stick of choice (portable USB HDD works aswell)
- A PC running at least a Windows Vista kernel (Windows 7, 2008 – WinPE versions of these also work)
- Administrator privileges on the machine.
With the introduction of Vista and later kernels making USB storage bootable for windows became so much easier. Before (win Xp/2003 days there were specialized tools that major hardware vendors had, as far as I know). Why this is was not working in XP/2003?
The reason is that 5.x Versions of Windows did not mark a USB device as hard disk storage device, and you could not use “diskpart” command line utility, to mark partitions as active, what you need to get WinPE to boot.
Diskpart is a pretty powerful disk partitioning utility that comes free with Windows, can be used to manage hard disks when you lack a GUI or are a scripthead like me . What is great about it is that it’s also included in WindowsPE which can make WinPE a one stop shop for disk management up to a certain point.
The steps we will go through are:
- Use Diskpart to list all system disks and select our USB stick
- Partition USB stick as desired and mark a specific partition as active
- Copy WindowsPE files to the USB stick
Select system drive for WinPE boot
First step is open up your command prompt running it as Administrator. Type diskpart at the prompt and hit enter. After a few seconds you get a command prompt headed by “DISKPART>”.
At the prompt type list disk - this will list all available physical drives. This next step is CRUCIAL. Pay extremely good attention to the information shown. The command will list the disks on your system. Identify which on the disks listed there is your USB stick, use the “Size” parameter to figure it out. If you are still not sure what type of disk you have selected do following:
select disk [enter #] <<enter a disk number from the listing you did>> detail disk
The output should say your select disk is TYPE:USB. If you get TYPE:ATA, list the disks again and select another number!
DISKPART> detail disk <DEVICE NAME> Disk ID: 2EA32EA2 Type : USB Status : Online Path : 0 Target : 0 LUN ID : 0 Location Path : UNAVAILABLE Current Read-only State : No Read-only : No Boot Disk : No Pagefile Disk : No Hibernation File Disk : No Crashdump Disk : No Clustered Disk : No Volume ### Ltr Label Fs Type Size Status Info ---------- --- ----------- ----- ---------- ------- --------- -------- Volume 6 G NTFS Removable 7679 MB Healthy
Repeat the “select disk #” and “detail disk” until you find your USB device that you wish to make bootable. To see which disk you selected run:
DISKPART> list disk Disk ### Status Size Free Dyn Gpt -------- ------------- ------- ------- --- --- Disk 0 Online 186 GB 0 B * Disk 1 Online 7680 MB 0 B
Up to this point we’ve done nothing to the usb device, but I hope you have a backup /don’t care about the data on, because it will be gone in the next step…
Prepare disk and mark as active
With the proper selected disk we will wipe all partitioning data from it, create a single partition, format it as NTFS, give it a drive letter, and mark the partition as active.
!WARNING!: The following will wipe your device, so make sure the selected disk it is the correct one. (use list disk and look for the “*” to see which disk is selected)
clean create partition primary format fs=NTFS quick assign letter=U active exit
All of these commands will echo a response that they ran successfully, once done type exit to leave the diskpart context and let’s copy the WinPE files to disk.
Copy Windows PE Files
Ok, time for a little linking to my previous posts (post1 and post2). In these posts I discussed how to make a Windows PE boot Disk. IF you followed that tutorial (or similar ones on the internet) you will probably be stuck with a folder called ISO in the <PATH>\WinPE_x86\ folder. Also you should left with the ISO file. BOTH of them are good for this next step which is “Copy all the files from either the ISO folder or the ISO IMAGE you built to the root of the USB partition” (make sure you copy all files and folders including hidden ones). Yep, It’s that easy
Once you are done safely remove the USB device from the PC and attempt to boot from it. It should plain and simple work (provided your bios can boot from USB disk and you configured it to boot from USB disk properly). If your ISO image was working your USB stick should also be working.
I hope this was helpful and if you have feedback it is always welcome.
How to create a Windows PE Disk (part 2)
Mar 5th
It is time for part 2 of this guide to making yourself a Windows PE disk. You can read more about the first part here. This post we will cover following: how to integrate drivers, add 3rd party/applications/files to your image, unmount the image and burn it to an ISO file. I also want to say that this is a scripted approach, and all data and scripts are in the E:\PE path in this guide. The scripted approach will come in handy when you are doing tens of rebuilds of the image because a certain driver will not integrate, or a registry file modification does not work.
Integrate Drivers into WinPE Image
First thing on the agenda here is to get the actual drivers you want to integrate into the Image. For most use cases it is enough to integrate Storage and Network drivers, and perhaps Chipset drivers. You also need to take into account the WinPE version you are building, in this guide, we build a x86 WinPE Image so my focus was on x86 drivers for Windows 7/ 2008 /2008 R2. Now go out and grab those CD’s or vendor provided tools (Hyper-V Integration components or VMware Tools).
Some vendors ship other applications along with drivers, you don’t need the extra files most of the time, because WinPE doesn’t know how to use them most of the time. From the drivers in the list WinPE needs *.inf, *.cat and *.sys files corresponding to each driver you want to integrate and ANY other file specified in the *.inf file. Be patient with this process, as it can be sometimes painstaking and will cause you to rebuild your image until you get it right, until you find all the drivers and files you need
Let’s take the example of VMware Tools for vSphere. If you want your WinPE to boot into vSphere and be able to see your storage adapters and network cards you need to integrate the drivers from VMware Tools.
Step1: On a VM running Windows 2008/ Windows 7 on vSphere start an interactive VMwareTools Install.
Step2: Install your VMware Tools and reboot VM. Now take a look in %programfiles%\vmware\vmware tools\drivers\ – driver heaven! Copy the needed folders from here into a folder called “E:\PE\Drivers\ESX_40″ (e:\PE is the location where we run our WinPE imaging process).
For other drivers you may need to take a different approach. I will just share from my experience. Drivers can be in *.cab cabinet files, in *.zip files, inside MSI files, which you kinda have to install to get to (see vmware tools case), even install a driver and then look in device manager where the device driver exists and search for a similarly named *.inf, sys and *.cat file and all the other files referenced in the *.inf file.
When you have all your drivers run this as administrator from a command prompt:
c: cd \ cd "%PROGRAMFILES%\windows aik\Tools\x86\Servicing" DISM /image:e:\pe\winpe_x86\mount /Add-Driver /driver:e:\PE\Drivers\ /recurse
Here you run the DISM tool using /add-driver switch, /driver specifying where the drivers are located, and /recurse to make it look in all subfolders in e:\PE\Drivers. This is one of the sweetest things about the DISM, is that it can recursively search for drivers (in WinPE 2.0 you had to have 1 command per folder containing drivers).
The output of the command should look like this:
As you can see DISM searched the folder and found 84 drivers (inf files that he can integrate). I had 85 inf files inside that folder, one failed, and you see DISM threw and error. This is however just a “pre-flight” check, as there can still be errors during the actual integration:
As you can see in this screenshot, DISM could not integrate some of the drivers and pointed to the DISM log file. This file can be found in %WINDIR%\Logs\DISM\dism.log.
For those that just want to test their driver integration skip the next step.
Adding Custom Scripts/Applications to the Image
In an earlier post, I showed how to mount the WinPE Image. The Image was mounted under “E:\PE\winpe_x86\mount”. If you take a look in this folder you will notice a folder structure resembling a windows install…well that is exactly what it is – all Windows PE files unpacked, as they would look like if booted with the image. This means you can add files under %windir%\system32 of the image (in our case Windows\e:\pe\winpe_x86\mount\windows\System32) and you would be able to execute them as %windir%\system32 is in the %path% environment variable of the Windows OS. Note that not all apps run under Windows PE, sometimes it is a matter of trial and error.
So it is just a matter of copying the files you need from a path, let’s say “e:\PE\CustomApps\” to wherever you want in the folder structure “e:\PE\winpe_x86\mount\”. Use a manual copy or do an xcopy like this for example:
xcopy /y /r /F E:\PE\CustomApps E:\PE\winpe_x86\mount\Windows\System32
It is a little known fact about Windows PE that it has a batch file called “startnet.cmd”. This file includes a command “wpeinit”. wpeinit is an executable that is run when WindowsPE boots on your system (more info here). While I don’t care much about wpeinit itself, I do care about startnet.cmd. This file you can modify/overwrite at this point with a custom made startnet.cmd that can start other scripts, check IP connectivity anything you need to do with your WinPE boot disk. Similar to putting custom apps on WinPE you can do this:
xcopy /h /Y /R /F "E:\PE\CustomScripts\startnet.cmd" "E:\PE\winpe_x86\mount\Windows\System32\startnet.cmd"
I am stressing the importance of this file because, you can access it only at boot time and it is “hard-coded” into the WIM file (you cannot change it after you unmount the WIM and build the ISO afterwards). Therefore, since startnet.cmd cannot be altered after building the image, it could make sense to have startnet.cmd point to a file say, autorun.cmd, that you can put on the root of the ISO file for example. And there are many ISO editing tools,so changes to autorun.cmd are easier to make, for editing a WIM things are not so straightforward.
Still following this? Good, because the worst part is over
Unmount Image and burn to ISO
This last step is fairly easy. DISM has a parameter to unmount the image and commit the changes to the Image. If you remember in the beginning we copied boot.wim to winpe.wim. now we overwrite the existing boot.wim image with our serviced image. The commands below do just that:
c: cd "%PROGRAMFILES%\windows aik\Tools\x86\Servicing" ::commit changes to image and unmount Dism.exe /Unmount-Wim /mountdir:E:\PE\winpe_x86\mount /commit copy E:\PE\winpe_x86\winpe.wim e:\pe\winpe_x86\ISO\sources\boot.wim /Y
In the current state you have 2 options:
1. Copy the contents of the E:\pe\winpe_x86\iso folder to a bootable USB stick or make an iso file out of it. For now let’s focus on making a ISO file.
Microsoft delivered OSCDIMG with the WAIK, a utility that can create the bootable ISO for us.
cd \ C: cd "%PROGRAMFILES%\windows aik\Tools\x86" ::"-b" MUST BE next to path for etfsboot.com OSCDIMG -bE:\PE\winpe_x86\etfsboot.com -n -o E:\PE\winpe_x86\iso E:\PE\Current_ISO.iso
Please note the comment in the script, “feature” or bug you don’t need a space between -b and the etfsboot,com file.
This should have successfully built the image and you can burn it to a CD/mount it in a VM and enjoy a Microsoft Supported Windows 7 live CD . Before you go take a mental break from all this reading I just want to point out that Windows PE will crash if you run it on a system with insufficient memory.
Why? The boot disk creates a Ramdisk where he loads Windows PE. If there is not enough RAM memory (typically you have this issue on old hardware or VM’s) it will crash and simply not load. As a rule of thumb the machine using it should have at least 1.8 -2.0 the size of the ISO file as RAM available on the machine.
I hope this was helpful for others looking to use WinPE as boot disk and I appreciate any feedback you may have.
How to create a Windows PE boot disk
Feb 28th
Some time ago I worked a lot on this topic, and had a good system for creating Windows PE disk, but recently I had to adapt the workflow to changes in the WAIK (Windows Automated Installation Kit) introduced with Windows 7/2008 R2. I won’t go into details on what’s new, but as far as I am concerned it made the whole process quite a bit easier.
Maybe I should explain also what the rest of the Internet will probably tell you: “What is Windows PE?”
Answer: Windows PE is a lightweight version of the Windows operating system.
Why should you care about Win PE?
Answer: Well because you can…
-Access the NTFS shares, map drives and copy data over the network
-Format and partition disks and make bootable USB sticks
-Run admin tools like PsTools, NTPWEdit (password recovery)
-Run WMI and batch scripts to automate Windows 2003 Server / Xp installations
-Edit offline registry of a PC (e.g. change it’s IP address, start/stop services) when you can’t login to the OS for some reason.
It’s been around since Windows Xp (versions 1.x), but only since version 2.0 and now version 3.0 do have some real punch to them. In a way I think it is better than other Windows boot CD’s like Bart and the likes, because you can get MS support and it provides a “clean”,”supported” way to customize it over time with drivers and apps.
Here’s what we will do to get a working WinPE 3.0 boot image:
- Download WAIK and install it
- Copy files from the WAIK source folder.
- Mount the WIM image
- Add additional components to the WIM
- Integrate drivers into the image
- Add applications/other scripts/files to the image
- Unmount WIM Image
- Burn the image into an ISO file
WAIK Installation
In order to create this WinPE disk you need the WAIK (Windows Automated Installation Kit). This is for WinPE version 3, there are more versions of this WAIK out there, and this tutorial only works for WinPE 3.0.After you’ve downloaded it simply mount and install it. Should you have any issues with the installation (i have some trouble in the days of 2.x) check the contents of the ISO image there should be some file called “wAIKX86.msi” that you can use to launch the installation on a 32b OS, same goes for 64b OS. Installation is pretty simple, next next next.
Copy Required files
I should get this out of the way from the start. These steps are a scripted approach to make a boot disk. While I’m sure there is a GUI somewhere, while I was building my workflow using it was impractical, as I needed a way to reduce user errors and test quickly different approaches in an automated way.
To make this whole process easier make sure you add following paths to your %PATH% environment variable: %PROGRAMFILES%Windows AIK\Tools\PETools
Also create a folder where you have full administrator rights (in this post i use “e:\PE”). Save yourself some headache and use a shortname with no spaces. Needless to say you need to be an administrator on the computer you are using for this task and all of the commands need to be run from an elevated command prompt.
First step is to get ourselves all the files that we will need to make the image. These files are installed by the WAIK installer, and MS also conveniently provides a batch script that copies everything. From the elevated prompt run this:
start /wait cmd /c copype.cmd x86 e:\PE\winpe_x86
We use copype.cmd located under %PROGRAMFILES%Windows AIK\Tools\PETools. The script copies the x86 WinPE files to a customization directory.
Mount the WIM Image
Next we use DISM to mount the boot.wim image and begin servicing it. MS introduced this tool called DISM (Deployment Image Servicing and Management) as a single point of servicing the WinPE image.
c: cd "%PROGRAMFILES%\windows aik\Tools\x86\Servicing" Dism.exe /mount-wim /wimfile:"E:\PE\winpe_x86\winpe.wim" /index:1 /mountdir:"E:\PE\winpe_x86\mount" ::list packages installed - do not use quotes in image name dism /image:E:\PE\winpe_x86\mount /Get-Packages
We mount the image (a .WIM file) to the “e:\PE\winpe_x86\mount” directory. Once we mount the image we can do anything with the files inside the mounted image. I also did a listing of the packages inside the image so you can see what’s inside it. We just have the basics, a language pack, and the foundation. When using dism with /image be careful to not use quotes in the image name.
Add WIM components
Now we can add so called packages to the WIM image. These packages are extra features you may want your image to have, like WMI support, MDAC support, Windows 7/Windows 2008 server setup screens, support for other languages, etc. For a list of the available package for the WinPE x86 version look in %PROGRAMFILES%Windows AIK\Tools\PETools\x86\WinPE_FPs.
c: cd\ cd "%PROGRAMFILES%\windows aik\Tools\x86\Servicing" ::adding packages Dism /image:e:\PE\winpe_x86\mount /Add-Package /PackagePath:"%programfiles%\Windows AIK\Tools\PETools\x86\<add-your-package-filename-here>" ::add here any other packages you need
For enabling vbs and WMI support add these packages: winpe-scripting.cab, winpe-wmi.cab, winpe-mdac.cab, winpe-hta.cab. I’ve added MDAC and HTA packages to the list in case your vbs scripting requires them, There are corresponding packages in the en-us folder, add them aswell, with the syntax above. This is how the output should look like once you run all commands for adding the packages:
In the end run dism /image:E:\PE\winpe_x86\mount /Get-Packages to list the packages you installed.
This covers half the process I described at the start of this post. in the coming days the second part of this tutorial will be finished. I hope you found it useful, have a great week!
Removing specific message(s) from multiple Exchange 2007 mailboxes
Feb 21st
I seem to be doing quite a bit Powershell scripting these days, and some of related to MS Exchange 2007. One issue we had recently was that loose permissions on Distribution Lists with hundreds of users + too much spare time for some users generated a lot of unwanted message traffic. I don’t want to discuss prevention measures like restricting who can send emails to big DLs or using Microsoft AD RMS to restrict what can be done with emails. Our goal here is to clean up the mess 
.
Essentially you can get some info about the message and mailboxes, and use it with Export-mailbox to remove the data. That is how I initially found this link, but what is not written there is that you need to have all the prerequisites for running export-mailbox, and also running it on hundreds of mailboxes may take a while. I decided to do it my way,by building on what I found on that blog.
This is “Mass Remove message(s) from mailboxes – My Way”.Depending on your situation you can apply these steps multiple times:
- Identify the message that started it all
- Track the message on the Exchange Servers and compile a list of unique recipients of the message.
- Remove the message from the offended mailboxes (there may be special requirements to perform the task, see here)
Identify Message
Getting the information should be pretty easy, someone probably forwarded you the copy of the message(s) to be dealt with. You want to get this info as a minimum: subject,sender,date and time message(s) was/were sent. When you have enough info, open the Exchange Management Console > Tools > Message Tracking and from there identify which of the events represent the time the message originally arrived on the Transport Servers. For that event grab the “MessageID” Value. We will use this in the following steps to find all events relating to that specific messageID.
Track Message
Assuming the worst case scenario you have to do tracking across all Exchange Transport Servers, the speed of the process depends on how close to your Exchange Transport Role Servers you are running the tracking. I suggest you make sure this process runs in the same LAN as the Exchange, especially the export-mailbox part. Anyhow, to get all messages sent by “baduser@foo.com” across all transport servers in your Exchange run this:
$TrackingLogResults = get-transportserver | where {$_.Name -like "<optional filter>"} | foreach-object {Get-MessageTrackingLog -EventId DELIVER -MessageID <MessageID from Step1> -ResultSize Unlimited -server $_}
- Get-TransportServer gives you all the transport servers in the organization
- Where clause filters the servers list, you can leave it out, it is helpful if your HUB transport servers are named in a specific way, and you know the message did not leave the organization, so you can exclude a search on the Edge Servers.
- Foreach-Object cycles through all servers and performs the search
- Get-MessageTrackingLog searches each transport server tracking log for DELIVER Events that correspond to messages with that specific MessageID. It returns unlimited results. The server that is being searched is piped from the Foreach cmdlet.
- If you run the last cmdlet without the EventID filter, you will get lots of other EventID’s like fail,send,receive,routing,expand. You just need deliver, DELIVER is important because it basically says “OK, this message passed all of my checks I am now sending it to the Mailbox Server so it can submit it to the mailbox store”, so you get a list of just the actually affected mailboxes.
This may take a while to run. Once it is finished we have to get the list of people that the message was sent to. The easy answer would be “why not just do $TrackingLogsResults | select-object Recipients and pipe it along to something else?”
Well you can do that, but in some cases Recipients means actually a bunch of other addresses, and each recipient may appear multiple times in the entire list.
e.g. – this could be a list returned by the “easy” command
{john@foo.com}
{John@foo.com,Jane@foo.com}
{Jill@foo.com,Josh@foo.com,Jake@foo.com}
Having duplicates is inefficient, everything will take longer in next steps. What I wanted was to have a list without duplicates, plus I get to show you some more “nice” scripting stuff
Compile Recipients List
I spent quite some time figuring this out, so someone out there better find it useful . The next step involved a “google shovel” to “dig up” how to break up those objects into one big list. Then the plan was to have a list that just had the unique email addresses – ideally. So here’s the “magic”:
$RecipientsExpanded = @()
$RecipientsExpanded = $TrackingLogResults | foreach-object {$RecipientsExpanded = $RecipientsExpanded + ($_.Recipients)}
$RecipientsGrouped = $RecipientsExpanded | group-object
$UniqueRecipients = $RecipientsGrouped | select-object Name | sort-object -property name
- We created a blank array object that will host all recipients addresses in “expanded form”.
- For each result from the TrackingLog we added the array ($_.Recipients) to the $RecipientsExpanded array. At the end of this we have a single array with all the addresses, each an individual element in the array.
- The Group-Object cmdlet is used to group all addresses by their name and in the end you have the list of unique email addresses.
Actually remove offending messages
Please see this link if you are planning to export the messages to PST. What is left to do is to take a page from the MSExchangeTeam blog and run get-mailbox| export-mailbox combo, only we are doing it on a reduced scale, only on the mailboxes that need it, that why I went through all the trouble of making that list!
$MailboxesList = $UniqueRecipients | foreach-object {
$Filter = "PrimarySmtpAddress -eq '"+$_.Name+"'"
get-mailbox -ignoredefaultscope -resultsize unlimited -Filter $Filter}
The code above handles this task for forests with child domains. I covered reasoning and use of -Ignoredefaultscope and -Filter in a previous post.
#get current admin UserPrincipalName $Admin = [Security.Principal.WindowsIdentity]::GetCurrent().Name #elevating the administrator's account to fulll access over all affected mailboxes Add-MailboxPermission $MailboxesList -AccessRights FullAccess -User $Admin export-Mailbox -Identity $MailboxesList –ContentKeywords <enter part of message body> -Recipients <add recipients list> –TargetMailbox admin_ –TargetFolder "RecoveredEmails" –DeleteContent
- The final step grants the admin user full access over the mailbox. The account being granted that right is $Admin, the account under which the script is running, it contains the UserPrincipalName of that account.
- You also need to have admin rights on the “TargetMailbox” and the “TargetFolder” should also exist beforehand.
- We export the offending message(s) using Export-Mailbox. Here it is important to be very careful and make the filtering as strict as possible, since here you cannot remove a message based on the MessageID, so you could end up removing many more messages. Refer to the documentation for export-mailbox, for all available switches for this purpose.
After you run the last command get ready for some really long waiting, as it goes through all the mailboxes. Once it is finished, remove your permissions from those mailboxes.
Remove-MailboxPermission $MailboxesList -AccessRights FullAccess -User $Admin
Phew this was a long post, but validating everything I explained here, took a while. The post is also packed with bits and pieces that can be your building block for other Exchange Shell scripts. I tried to show you how to take Exchange TrackingLog data and build a list of unique recipient addresses that you can use to filter out an unwanted message you tracked in the logs, and do that using export-mailbox commandlet. If you have any feedback/corrections/omissions please feel free to leave a comment.
Happy Scripting!
