Logging Data Using Splunk – Part 2 – Deploying the Forwarder on Windows

Last post I showed you how to install the Splunk Indexing Server and make it listen for data, by enabling receiving of forwarded events. That’s all very nice, but someone needs to actually send data to that port, for Splunk to index it. We are going to focus on the Windows deployment of a Forwarder, but some of the steps here are applicable, in essence to a Linux forwarder:

  • Fulfill Installation Prerequisites
  • Install Forwarder
  • Configure Forwarder

Installation Prerequisites

Some of the information mentioned here is also mentioned in the relevant Splunk documentation. I’m assuming you want Splunk to run on a domain network, and also it running on domain controllers. Essentially Splunk runs in the system using 2 services “Splunkd” and “Splunkweb”. The forwarder only needs “Splunkd” service to run. With that in mind, here is what you need to run Splunk on Windows Servers:

  • Splunk Forwarder version must be at most equal to the version of the Indexer, so your Forwarders cannot be more advanced than the Indexer. I have not tempted fate to see what breaks otherwise 😉
  • Make sure you install 32bit Splunk on 32bit OS’s and 64bit on 64bit OS’s. Splunk says 64b version offers a lot of improvements, in light of people moving to Windows 2008 server, everyone should be happy.
  • You will need the Splunk MSI package, get it from here.
  • You need a domain account that Splunk Services can run under. That account must be a Local Administrator on Servers where Splunk Forwarder will be installed. If you are focused on security, check documentation link above, for minimum requirements. You can use a GPO to enforce these settings as well.
  • To push Splunk Forwarder remotely /via script make sure the account used to run the installation can be elevated to Administrator (aka UAC does not break the install – for Windows Server 2008/ Windows 7); this is especially important in this tutorial since this will be a scripted install.
  • Make your life easier and keep the Splunk.msi on a network share along with any installation scripts. Also secure that share as best you can, since some data is in clear text.

Install the Forwarder

For installing the forwarder we will make a command line install. The installer allows more customization via the CLI than via all the install menus. For reference you can take a look here for all CLI switches, but note that not all switches work as advertised. There are a lot of CLI switches designed to customize Splunk upon on installation, but since some of them do not work and the fact that Splunk can be customized after the installation, I used only switches that worked and I could not configure after the installation. Here’s the magic, that you need to put on a Windows NT batch file (“.bat”) and run it.

::Stop all splunk services
net stop splunkd
net stop splunkweb
::Remove all splunk versions
start /wait MsiExec.exe /uninstall {60ad9785-709f-4b4d-ac19-91cbe0ab7614} /passive
start /wait MsiExec.exe /uninstall {a7579aaa-db6b-46ce-90ca-d8f553481bcc} /passive
start /wait MsiExec.exe /uninstall {2c0fae08-7c9c-40f9-ba21-82a2aad07f0d} /passive

::Map drive to splunk install path
net use /delete S:
net use S: <map network path of splunk executable>

::Execute installation string, minimal configuration
start /wait msiexec.exe /i S:\splunk-4.0.9-74233-x86-release.msi INSTALLDIR="%ProgramFiles%\Splunk" RBG_LOGON_INFO_USER_CONTEXT=2 IS_NET_API_LOGON_USERNAME="<domain\SplunkServiceUser>" IS_NET_API_LOGON_PASSWORD="<Password>" LAUNCHSPLUNK=0 AUTOSTARTSERVICE_SPLUNKD=1 AUTOSTARTSERVICE_SPLUNKWEB=0 /passive

Breaking the code down really quick:

  • Stop splunk services, just to make sure. You can foolproof the code by also forcefully killing Splunk related processes.
  • Use the “uninstall current version” section to rid yourself of previous versions of Splunk. This will be a growing list of commands…because:

    • Important Note: The Installation ID of Splunk is different from 32b version to the 64b version, and from different 32b/64b versions, so make sure you get the Installation ID correctly from the registry or however you know. Reg key is here HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall
  • The specific parameters related to Splunk are as follows
    • Specify installation directory – INSTALLDIR
    • Specify how services run (Local System or Domain Account – we used Domain Account – RBG_LOGON_INFO_USER_CONTEXT=2 )
    • Specify credentials used (IS_NET_API_LOGON_USERNAME, IS_NET_API_LOGON_PASSWORD)
    • Specify what happens to splunk after installation is finished and status of the Splunk Services, we want splunk to do nothing and we don’t need Splunkweb (LAUNCHSPLUNK, AUTOSTARTSERVICE_SPLUNKD, AUTOSTARTSERVICE_SPLUNKWEB)
  • The last /passive switch is typical to the MSI installer, use /quiet if you prefer. Also you do not need to reboot after a Splunk Install/Uninstall.

Improvements to this piece of NT batch code? Yes we can

  • Remove any values from the Username and Password field and replace the parameters with:
    • IS_NET_API_LOGON_USERNAME=”%1″
    • IS_NET_API_LOGON_PASSWORD=”%2″
  • Save the script above to a batch file, SplunkDeploy.bat for example, and run the batch file like this:
    • SplunkDeploy.bat domain\SplunkServiceAccount reallyhardpasssword where you replace the bold text with your specific Splunk account credentials.
    • This ensures that no passwords are kept in clear text, quite a big no no considering this sort of account kind of owns all computers on the domain, one way or another.
  • At this point Splunk is installed and configured with the default settings. Notice what we have done now relates very little to the Forwarding role Splunk will have, this will be addressed in the Configuration section. As I don’t really like the default configuration, and since going into explaining why, requires another post and a bit much of reading attention I hope you will stay tuned for the sequel to Part 2, IMO the most complex part of the series 😉