How to create a Windows PE Disk (part 2)


It is time for part 2 of this guide to making yourself a Windows PE disk. You can read more about the first part here. This post we will cover following: how to integrate drivers, add 3rd party/applications/files to your image, unmount the image and burn it to an ISO file. I also want to say that this is a scripted approach, and all data and scripts are in the E:\PE path in this guide. The scripted approach will come in handy when you are doing tens of rebuilds of the image because a certain driver will not integrate, or a registry file modification does not work.

Integrate Drivers into WinPE Image

First thing on the agenda here is to get the actual drivers you want to integrate into the Image. For most use cases it is enough to integrate Storage and Network drivers, and perhaps Chipset drivers. You also need to take into account the WinPE version you are building, in this guide, we build a x86 WinPE Image so my focus was on x86 drivers for Windows 7/ 2008 /2008 R2. Now go out and grab those CD’s or vendor provided tools (Hyper-V Integration components or VMware Tools).

Some vendors ship other applications along with drivers, you don’t need the extra files most of the time, because WinPE doesn’t know how to use them most of the time. From the drivers in the list WinPE needs *.inf, *.cat and *.sys files corresponding to each driver you want to integrate and ANY other file specified in the *.inf file. Be patient with this process, as it can be sometimes painstaking and will cause you to rebuild your image until you get it right, until you find all the drivers and files you need 🙂

Let’s take the example of VMware Tools for vSphere. If you want your WinPE to boot into vSphere and be able to see your storage adapters and network cards you need to integrate the drivers from VMware Tools.

Step1: On a VM running Windows 2008/ Windows 7 on vSphere start an interactive VMwareTools Install.

Step2: Install your VMware Tools and reboot VM. Now take a look in %programfiles%\vmware\vmware tools\drivers\ – driver heaven! Copy the needed folders from here into a folder called “E:\PE\Drivers\ESX_40” (e:\PE is the location where we run our WinPE imaging process).

For other drivers you may need to take a different approach. I will just share from my experience. Drivers can be in *.cab cabinet files, in *.zip files, inside MSI files, which you kinda have to install to get to (see vmware tools case), even install a driver and then look in device manager where the device driver exists and search for a similarly named *.inf, sys and *.cat file and all the other files referenced in the *.inf file.

When you have all your drivers run this as administrator from a command prompt:

c:
cd \
cd "%PROGRAMFILES%\windows aik\Tools\x86\Servicing"
DISM /image:e:\pe\winpe_x86\mount /Add-Driver /driver:e:\PE\Drivers\ /recurse

Here you run the DISM tool using /add-driver switch, /driver specifying where the drivers are located, and /recurse to make it look in all subfolders in e:\PE\Drivers. This is one of the sweetest things about the DISM, is that it can recursively search for drivers (in WinPE 2.0 you had to have 1 command per folder containing drivers).

The output of the command should look like this:

As you can see DISM searched the folder and found 84 drivers (inf files that he can integrate). I had 85 inf files inside that folder, one failed, and you see DISM threw and error. This is however just a “pre-flight” check, as there can still be errors during the actual integration:

As you can see in this screenshot, DISM could not integrate some of the drivers and pointed to the DISM log file. This file can be found in %WINDIR%\Logs\DISM\dism.log.

For those that just want to test their driver integration skip the next step.

Adding Custom Scripts/Applications to the Image

In an earlier post, I showed how to mount the WinPE Image. The Image was mounted under “E:\PE\winpe_x86\mount”. If you take a look in this folder you will notice a folder structure resembling a windows install…well that is exactly what it is – all Windows PE files unpacked, as they would look like if booted with the image. This means you can add files under %windir%\system32 of the image (in our case Windows\e:\pe\winpe_x86\mount\windows\System32) and you would be able to execute them as %windir%\system32 is in the %path% environment variable of the Windows OS. Note that not all apps run under Windows PE, sometimes it is a matter of trial and error.

So it is just a matter of copying the files you need from a path, let’s say “e:\PE\CustomApps\” to wherever you want in the folder structure “e:\PE\winpe_x86\mount\”. Use a manual copy or do an xcopy like this for example:

xcopy /y /r /F E:\PE\CustomApps E:\PE\winpe_x86\mount\Windows\System32

It is a little known fact about Windows PE that it has a batch file called “startnet.cmd”. This file includes a command “wpeinit”. wpeinit is an executable that is run when WindowsPE boots on your system (more info here). While I don’t care much about wpeinit itself, I do care about startnet.cmd. This file you can modify/overwrite at this point with a custom made startnet.cmd that can start other scripts, check IP connectivity anything you need to do with your WinPE boot disk. Similar to putting custom apps on WinPE you can do this:

xcopy /h /Y /R /F "E:\PE\CustomScripts\startnet.cmd" "E:\PE\winpe_x86\mount\Windows\System32\startnet.cmd"

I am stressing the importance of this file because, you can access it only at boot time and it is “hard-coded” into the WIM file (you cannot change it after you unmount the WIM and build the ISO afterwards). Therefore, since startnet.cmd cannot be altered after building the image, it could make sense to have startnet.cmd point to a file say, autorun.cmd, that you can put on the root of the ISO file for example. And there are many ISO editing tools,so changes to autorun.cmd are easier to make, for editing a WIM things are not so straightforward.

Still following this? Good, because the worst part is over 🙂

Unmount Image and burn to ISO

This last step is fairly easy. DISM has a parameter to unmount the image and commit the changes to the Image. If you remember in the beginning we copied boot.wim to winpe.wim. now we overwrite the existing boot.wim image with our serviced image. The commands below do just that:

c:
cd "%PROGRAMFILES%\windows aik\Tools\x86\Servicing"
::commit changes to image and unmount
Dism.exe /Unmount-Wim /mountdir:E:\PE\winpe_x86\mount /commit
copy E:\PE\winpe_x86\winpe.wim e:\pe\winpe_x86\ISO\sources\boot.wim /Y

In the current state you have 2 options:

1. Copy the contents of the E:\pe\winpe_x86\iso folder to a bootable USB stick or make an iso file out of it. For now let’s focus on making a ISO file.

Microsoft delivered OSCDIMG with the WAIK, a utility that can create the bootable ISO for us.

cd \
C:
cd "%PROGRAMFILES%\windows aik\Tools\x86"
::"-b" MUST BE next to path for etfsboot.com
OSCDIMG -bE:\PE\winpe_x86\etfsboot.com -n -o E:\PE\winpe_x86\iso E:\PE\Current_ISO.iso

Please note the comment in the script, “feature” or bug you don’t need a space between -b and the etfsboot,com file.

This should have successfully built the image and you can burn it to a CD/mount it in a VM and enjoy a Microsoft Supported Windows 7 live CD :). Before you go take a mental break from all this reading I just want to point out that Windows PE will crash if you run it on a system with insufficient memory.

Why? The boot disk creates a Ramdisk where he loads Windows PE. If there is not enough RAM memory (typically you have this issue on old hardware or VM’s) it will crash and simply not load. As a rule of thumb the machine using it should have at least 1.8 -2.0 the size of the ISO file as RAM available on the machine.

I hope this was helpful for others looking to use WinPE as boot disk and I appreciate any feedback you may have.

Logging Data Using Splunk – Part 2 – Deploying the Forwarder on Windows

Last post I showed you how to install the Splunk Indexing Server and make it listen for data, by enabling receiving of forwarded events. That’s all very nice, but someone needs to actually send data to that port, for Splunk to index it. We are going to focus on the Windows deployment of a Forwarder, but some of the steps here are applicable, in essence to a Linux forwarder:

  • Fulfill Installation Prerequisites
  • Install Forwarder
  • Configure Forwarder

Installation Prerequisites

Some of the information mentioned here is also mentioned in the relevant Splunk documentation. I’m assuming you want Splunk to run on a domain network, and also it running on domain controllers. Essentially Splunk runs in the system using 2 services “Splunkd” and “Splunkweb”. The forwarder only needs “Splunkd” service to run. With that in mind, here is what you need to run Splunk on Windows Servers:

  • Splunk Forwarder version must be at most equal to the version of the Indexer, so your Forwarders cannot be more advanced than the Indexer. I have not tempted fate to see what breaks otherwise 😉
  • Make sure you install 32bit Splunk on 32bit OS’s and 64bit on 64bit OS’s. Splunk says 64b version offers a lot of improvements, in light of people moving to Windows 2008 server, everyone should be happy.
  • You will need the Splunk MSI package, get it from here.
  • You need a domain account that Splunk Services can run under. That account must be a Local Administrator on Servers where Splunk Forwarder will be installed. If you are focused on security, check documentation link above, for minimum requirements. You can use a GPO to enforce these settings as well.
  • To push Splunk Forwarder remotely /via script make sure the account used to run the installation can be elevated to Administrator (aka UAC does not break the install – for Windows Server 2008/ Windows 7); this is especially important in this tutorial since this will be a scripted install.
  • Make your life easier and keep the Splunk.msi on a network share along with any installation scripts. Also secure that share as best you can, since some data is in clear text.

Install the Forwarder

For installing the forwarder we will make a command line install. The installer allows more customization via the CLI than via all the install menus. For reference you can take a look here for all CLI switches, but note that not all switches work as advertised. There are a lot of CLI switches designed to customize Splunk upon on installation, but since some of them do not work and the fact that Splunk can be customized after the installation, I used only switches that worked and I could not configure after the installation. Here’s the magic, that you need to put on a Windows NT batch file (“.bat”) and run it.

::Stop all splunk services
net stop splunkd
net stop splunkweb
::Remove all splunk versions
start /wait MsiExec.exe /uninstall {60ad9785-709f-4b4d-ac19-91cbe0ab7614} /passive
start /wait MsiExec.exe /uninstall {a7579aaa-db6b-46ce-90ca-d8f553481bcc} /passive
start /wait MsiExec.exe /uninstall {2c0fae08-7c9c-40f9-ba21-82a2aad07f0d} /passive

::Map drive to splunk install path
net use /delete S:
net use S: <map network path of splunk executable>

::Execute installation string, minimal configuration
start /wait msiexec.exe /i S:\splunk-4.0.9-74233-x86-release.msi INSTALLDIR="%ProgramFiles%\Splunk" RBG_LOGON_INFO_USER_CONTEXT=2 IS_NET_API_LOGON_USERNAME="<domain\SplunkServiceUser>" IS_NET_API_LOGON_PASSWORD="<Password>" LAUNCHSPLUNK=0 AUTOSTARTSERVICE_SPLUNKD=1 AUTOSTARTSERVICE_SPLUNKWEB=0 /passive

Breaking the code down really quick:

  • Stop splunk services, just to make sure. You can foolproof the code by also forcefully killing Splunk related processes.
  • Use the “uninstall current version” section to rid yourself of previous versions of Splunk. This will be a growing list of commands…because:

    • Important Note: The Installation ID of Splunk is different from 32b version to the 64b version, and from different 32b/64b versions, so make sure you get the Installation ID correctly from the registry or however you know. Reg key is here HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall
  • The specific parameters related to Splunk are as follows
    • Specify installation directory – INSTALLDIR
    • Specify how services run (Local System or Domain Account – we used Domain Account – RBG_LOGON_INFO_USER_CONTEXT=2 )
    • Specify credentials used (IS_NET_API_LOGON_USERNAME, IS_NET_API_LOGON_PASSWORD)
    • Specify what happens to splunk after installation is finished and status of the Splunk Services, we want splunk to do nothing and we don’t need Splunkweb (LAUNCHSPLUNK, AUTOSTARTSERVICE_SPLUNKD, AUTOSTARTSERVICE_SPLUNKWEB)
  • The last /passive switch is typical to the MSI installer, use /quiet if you prefer. Also you do not need to reboot after a Splunk Install/Uninstall.

Improvements to this piece of NT batch code? Yes we can

  • Remove any values from the Username and Password field and replace the parameters with:
    • IS_NET_API_LOGON_USERNAME=”%1″
    • IS_NET_API_LOGON_PASSWORD=”%2″
  • Save the script above to a batch file, SplunkDeploy.bat for example, and run the batch file like this:
    • SplunkDeploy.bat domain\SplunkServiceAccount reallyhardpasssword where you replace the bold text with your specific Splunk account credentials.
    • This ensures that no passwords are kept in clear text, quite a big no no considering this sort of account kind of owns all computers on the domain, one way or another.
  • At this point Splunk is installed and configured with the default settings. Notice what we have done now relates very little to the Forwarding role Splunk will have, this will be addressed in the Configuration section. As I don’t really like the default configuration, and since going into explaining why, requires another post and a bit much of reading attention I hope you will stay tuned for the sequel to Part 2, IMO the most complex part of the series 😉