Reverse DNS Lookups for Host Entries

[Michael Wilde]

When Splunk indexes, by default is going to take the hostname/ip that exists directly in the logfile entry...

Often, you would like to have the IP address resolved to a hostname, or vice versa. With Splunk 4.0 came a cool feature called "Lookups". Lookups allow for the enrichment of events in Splunk with data from external sources. Those sources can be a static CSV file (HTTP error codes is a good example), or a python script that is called at searchtime and grabs data from whereever you need it to. The python script must take in a CSV data structure and spit a CSV data structure back to Splunk.

Little did we know, Splunk included a file in $SPLUNK_HOME/etc/system/bin/ called "external_lookup.py" which just happens to resolve hostnames and IPs. That python script takes two possible pieces of data "host" and "ip" -- and when you're sending data to it via the "lookup" search command or in the config files, the field name HAS to be "host" or "ip".

You can choose to run the lookup via a search command "| lookup dnsLookup ip AS host OUTPUT host as hostname" or wire it in to run automatically as i have done below.

Note: either choice, you need to make sure the lookup is defined in "TRANSFORMS.CONF" as below.

1. mkdir $SPLUNK_HOME/etc/APPNAME/lookups (where APPNAME = search, for example)
2. copy $SPLUNK_HOME/etc/system/bin/external_lookup.py to $SPLUNK_HOME/etc/search/lookups
3. create/add to $SPLUNK_HOME/etc/search/local/props.conf a stanza to drive the lookup

PROPS.CONF
[firepass_log] <--set this to the sourcetype you'd like the lookup to occur on.
#lookups will apply to this sourcetype automatically
LOOKUP-rdns = dnsLookup ip AS host OUTPUT host as hostname
# "dnsLookup" will be defined in props.conf. This syntax reads "run dnsLookup, send it an "ip" address which we are going to get from the "host" field in the Splunk event, then when the lookup occurs, return the "host" information back to Splunk in the form of new field attached to each relevant event, called "hostname"

#In this case, we are doing reverse DNS lookup (get hostname from address) If you wanted to do forward DNS lookup (get ip address from hostname) this entry will work
LOOKUP-fdns = dnsLookup host OUTPUT ip

# This syntax reads "run dnsLookup, send it an hostname from Splunk's "host" field by each event, then when the lookup occurs, return the "ip" address back to Splunk in the form of new field attached to each relevant event, called "ip"

4. create/add to $SPLUNK_HOME/etc/search/local/transforms.conf a stanza to define details about the lookup.

TRANSFORMS.CONF
[dnsLookup]
# this stanza name will be called by your entry in props.conf and IS case sensitive
external_cmd = external_lookup.py host ip
fields_list = host, ip
# fields_list is the list of fields that will come back from the script and end up in your event. If you want these fields renamed, in your props.conf (As we did above), you can rename them with OUTPUT (field) as (newfieldname).

Check it out!