"How are these transactions linked together... by a field called "ID"? If so.. just build them with the field ID, and then use one of the MV commands to extract a field with success or failure in it. Paste some samples and…"
The dojo of Splunk. Learn, share, teach, mentor.
Videos
Latest Activity
Nikita posted a discussionCount failures and success via transaction
Hi,I'm a new in Splunk so sorry for the stupid questions.I want to calculate failures in logs.For example we have request log and response log."request" OR ("fail" OR "response") |transaction startsWith=("request") endsWith=("fail" OR "response") maxpause=5s keepevicted=false maxspan=25s id |eval Failure=if(searchmatch("fail"),1,0)| eval Success=if(searchmatch("response"),1,0) | stats count(Failure) as FailureCount, count(Success) as SuccessCount | table FailureCount SuccessCountThat query…See More
Handling Inputs - Blacklisting
While we all love Splunk's ability to just recursively eat entire directories of files in real time, there are some cases where you want to give splunk a "whoooa nelly.. not so fast"
For example: I've decided i want to monitor the /var/log directory on my Mac laptop, but i really don't need those old rolled over GZipped logfiles.
Normally, you just add an entry to the $SPLUNK_HOME/etc/system/local/inputs.conf to eat a directory, like this:
[monitor:///mnt/logs]
disabled=false
In my case, i'd like to have that monitor cruise through my directories and eat everything but files that have the ".gz" extension. To do that, just add a "_blacklist" entry in that stanza that contains the regular expression (regex) that matches the files you want. In our case, my inputs.conf will now look like this:
[monitor:///var/log]
_blacklist = \.gz$
Of course if i had multiple extensions i wanted to block, we could add an "or" regex, like this:
_blacklist = \.(txt|gz|tgz|bz2)$
Note: You can also use the syntax "_whitelist" which will disallow everything BUT what matches the whitelist's regex.
More info is over here on Splunk's docs as well
For example: I've decided i want to monitor the /var/log directory on my Mac laptop, but i really don't need those old rolled over GZipped logfiles.
Normally, you just add an entry to the $SPLUNK_HOME/etc/system/local/inputs.conf to eat a directory, like this:
[monitor:///mnt/logs]
disabled=false
In my case, i'd like to have that monitor cruise through my directories and eat everything but files that have the ".gz" extension. To do that, just add a "_blacklist" entry in that stanza that contains the regular expression (regex) that matches the files you want. In our case, my inputs.conf will now look like this:
[monitor:///var/log]
_blacklist = \.gz$
Of course if i had multiple extensions i wanted to block, we could add an "or" regex, like this:
_blacklist = \.(txt|gz|tgz|bz2)$
Note: You can also use the syntax "_whitelist" which will disallow everything BUT what matches the whitelist's regex.
More info is over here on Splunk's docs as well
Views: 143
Replies to This Discussion
-
Permalink Reply by Rob Jahn on -
I have a related requirement and I struggling with ....
#1. Here are series of logs in same folder but on multiple servers. I would like each file to have a unique sourcetype.
- from one server. note two sets folder names with A and B in the name
/apps/esrapp1/wls10/user_projects/domains/esrsDomainA_prd/drm.log <-- have sourcetype = esrs_drm
/apps/esrapp1/wls10/user_projects/domains/esrsDomainA_prd/esrs-portal.log <-- have sourcetype = esrs-portal.log
/apps/esrapp1/wls10/user_projects/domains/esrsDomainB_prd/drm.log <-- have sourcetype = esrs_drm
/apps/esrapp1/wls10/user_projects/domains/esrsDomainB_prd/esrs-portal.log <-- have sourcetype = esrs-portal.log
Tried this but the only the first log gets indexed
inputs.conf
[monitor:///apps/esrapp1/wls10/user_projects/domains/.../drm.log]
disabled = false
sourcetype = esrs_drm
index = esrs
[monitor:///apps/esrapp1/wls10/user_projects/domains/.../esrs-portal.log]
disabled = false
sourcetype = esrs_portal
index = esrs
Also tried this, but it did not work either
inputs.conf
[monitor:///apps/esrapp1/wls10/user_projects/domains/.../*.log]
disabled = false
index = esrs2
props.conf
[source::.../drm.log]
sourcetype = esrs2_drm
[source::.../esrs-portal.log]
sourcetype = esrs2_portal
#2 series of files that can all be the same sourcetype but both folder and file names vary
- from one server there are 4 files in folders like this.
- folder and file names with A and B in the name and the server number e.g. 03,04,05, etc.
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_core_vm02ms_1/esrsA_core_vm02ms_1.log
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_vm02ms_1/esrsA_vm02ms_1.log
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_core_vm02ms_1/esrsB_core_vm02ms_1.log
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_vm02ms_1/esrsB_vm02ms_1.log
- pattern repeats on other servers, but the number changes by server e.g. 03,04,05, etc.
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_core_vm03ms_1/esrsA_core_vm02ms_1.log
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_vm03ms_1/esrsA_vm02ms_1.log
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_core_vm03ms_1/esrsB_core_vm02ms_1.log
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_vm03ms_1/esrsB_vm02ms_1.log
Tried things like this but never got to work
[monitor:///apps/esrapp1/logs/.../esrs_*ms.log]
_whitelist = esrs_(core_vm\d\d|vm\d\d)ms_1.log
disabled = false
sourcetype = esrs2_weblogic
index = esrs2
#3 simular to #2, but include a file (access.log) from one subfolder but not the other
- from one server there are 4 files in folders like this.
- folder names with A and B in the name and the server number e.g. 03,04,05, etc.
- all files names are access.log
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_core_vm02ms_1/access.log <-- skip this one
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_vm02ms_1/access.log <-- keep this one
/apps/esrapp1/logs/esrsDomainB_prd/esrsB_core_vm02ms_1/access.log <-- skip this one
/apps/esrapp1/logs/esrsDomainB_prd/esrsB_vm02ms_1/access.log <-- keep this one
- pattern repeats on other servers, but the number changes by server e.g. 03,04,05, etc.
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_core_vm03ms_1/access.log <-- skip this one
/apps/esrapp1/logs/esrsDomainA_prd/esrsA_vm03ms_1/access.log <-- keep this one
/apps/esrapp1/logs/esrsDomainB_prd/esrsB_core_vm03ms_1/access.log <-- skip this one
/apps/esrapp1/logs/esrsDomainB_prd/esrsB_vm03ms_1/access.log <-- keep this one
This help file indicates that you can't combine file and directory wildcarding and if one monitor with a wildcard finds a folder match then it will ignore subsequent monitor ones that find the same folder.
http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputp...
I've experimented with a variety of settings in inputs.conf and props.conf but still not getting what I want. Any suggestions would be appreciated.
© 2012 Created by Michael Wilde.
