Searching hosts matching regex and not matching regex

[Colin Dick]

Hi there,
  New to Splunk, please bare with me ;).
 
  I have all my hosts set to IP addresses.  I want to search and get a list of all the hosts matching a certain pattern and then another search to get a list not matching the pattern.
 
search host=(regex 10.1.xxx.2 or 10.2.xxx.2) | count host by host
search host!=(regex 10.1.xxx.2 or 10.2.xxx.2) | count host by host
 
  I am hoping to be able to have output look similar to:
 
10.1.123.2
10.2.123.2
 
  And:
 
192.168.1.2
10.5.6.7
 
  Thanks for helping a newbie get rolling.  Learning how to format splunk searches looks like it is gonna take alot of time.  I have gone through the Splunk Tutorial.  Are there other good tutorials that you would recommend?
 
--
Colin

[Skeeter Murphy]

Try the following... also, you don't put 'search' at the beginning of a search phrase unless it's later in the pipeline.

 

host="10.1.*.2" OR host="10.2.*.2" | dedup host | table host

NOT host="10.1.*.2" NOT host="10.2.*.2" | dedup host | table host

 

There is an implicit AND unless you say OR.

The 'dedup' gives only one entry for each host value.

The 'table' says to only show the host field and nothing else.

 

You should also check out http://answers.splunk.com

 

Welcome to splunk. Enjoy.

 

skeeter

 

[Colin Dick]

Thanks Skeeter for reminding me about wildcards.  It helped initially.

 

Still working on some searches.  The wildcard search is now too generic.  I want to be able to determine if I have any unknown hosts.  I currently have 2 hosts 1.2.3.202 and 1.2.3.210.  So, I can search for 1.2.3.2*.  However, I want to ensure that I don't have any host data from 1.2.3.22 or 1.2.3.211.

 

I know I could be specific with host=hosta host=hostb, however, as I add hosts, my subset will grow and I would like to address it with a regex?  The regex 1\.2\.3\.2(02|10) matches, so, I have tried something like:

 

host=1.2.3.2* | rex (?i) (?P<NEWHOST>1\.2\.3\.2(02|10)) | dedup NEWHOST | table NEWHOST

 

However:

I am not sure if I am using the rex syntax properly.

The | symbol does not seem to work as OR within the search string but is interpreted as a pipe.

Not sure how I would take a negative of the regex so I could display results of non matching hosts.

 

Thanks in advance for any input.

 

--

Colin

[Michael Wilde]

There are a couple of other simple ways to approach this.  

 

You might consider using "tags".   I suspect there's something important about certain hosts, ex (1.2.3.202 might be a webserver, and 1.2.3.210 might be an appserver).  You can tag the host field for each ip address, and then you would search on "tag::host::webserver OR tag::host::appserver".

 

Now.. if you have another system that drives the list of hosts you'd like to search on, you can have that system (or yourself) via a script, create CSV and use a "lookup".  Create a CSV file that contains fields.

Example:

 

MYHOSTS.CSV

host,location,role

1.2.3.202,sheboygan,firewall

1.2.3.210,sheboygan,proxy_server

1.2.3.211,austin,file_server

 

Properly define this as a lookup table in splunk, and what i like to do is use the "inputlookup" search command as a subsearch.  What we're gonna do is create a subsearch that reads in the CSV at search time, looks for the two hosts from sheboygan, takes the output and sends it to an outer search. (sounds complicated, but its not).

 

First, get the sheboygan hosts

"| inputlookup myhosts.csv  | where location="sheboygan"| fields host"  -- that will give us a list of two hosts we care about. Great.. but not quite.  Now, lets use the output of this.. as input to a search that looks for "error messages" from our sheboygan hosts in the last 15 minutes.  That's done like this

 

"error minutesago=15 [  | inputlookup myhosts.csv  | where location="sheboygan"| fields host ]

 

The [ inner search ] is ran first, the list of hosts comes back and is sent to the outer search, and how your CSV drives the list of hosts you want to be searched.

 

Hopefully those ideas might help.

 

[Ralph Avery]

I think this is what you want...

* | regex host="(?=\b10\.[12]\.123\.2\b)" | stats count by host | sort by host
* | regex host!="(?=\b10\.[12]\.123\.2\b)" | stats count by host | sort by host