The dojo of Splunk. Learn, share, teach, mentor.
Videos
Latest Activity
Michael Wilde replied to Nikita's discussion Count failures and success via transaction"How are these transactions linked together... by a field called "ID"? If so.. just build them with the field ID, and then use one of the MV commands to extract a field with success or failure in it. Paste some samples and…"
Dave P's Discussions
Splunk skipping input files & Order of precedence
5 Replies
Hi,So, I think I'm missing something obvious here. a 2 part question.1) I have one of several inputs defined as:monitor:///data/logs]disabled = falsehost_segment = 4index = defaultsourcetype =…Continue
Tags: operations, of, order, files, inputs
Started this discussion. Last reply by Dave P Jan 3, 2010.
Optimizing searching over indexing
5 Replies
Hopefully, this will be first of many discussions I'll be part of. Found SN yesterday, very cool.I've got a single host handling both Splunk indexing and searching. I'd like to give searching…Continue
Started this discussion. Last reply by Dave P Jan 5, 2010.
Dave P's Page
Gifts Received
Dave P has not received any gifts yet
Latest Activity
Dave P replied to Dave P's discussion Optimizing searching over indexing"I hear that. I try to live in 64-bit land whenever possible.
When the real production Splunk gear arrives, it'll be 64-bit OS installation, so I'm hoping that I see some improvement in speed over equivalent hardware running a 32-bit…"
Michael Wilde replied to Dave P's discussion Optimizing searching over indexing"Dave... I deleted your reply on accident... there was a spammer in here.
Yes. Splunk does take advantage of 64bit in a MAJOR WAY!... When indexing on a 32bit machine, splunk can store its "buckets" in a max of 200MB per bucket--meaning…"
Dave P replied to Dave P's discussion Splunk skipping input files & Order of precedence"Aaaaand Splunk Support has come through again. Below is the solution that worked nicely, though it seems there should have been an easier way to do this.
in…"
Dave P replied to Dave P's discussion Splunk skipping input files & Order of precedence"There are no entries in the "Most recently ignored files" search results for the past 24 hours (or even the past 72 hours). So, at least Splunk is finding everything, even if it is not sourcetyping it correctly.
I found the same…"
Dave P replied to Dave P's discussion Splunk skipping input files & Order of precedence"I'm getting closer, I think.
I did what you suggested, created a separate entry for the maillog stuff in the props.conf, though it ended up looking like this:
[source::/data/logs/...]
sourcetype =…"
Alexander Szoenyi replied to Dave P's discussion Splunk skipping input files & Order of precedence"Hello,
Point 1
Go to the Search App -> Status -> Inputs Activity.
There you can find the "Most recently ignored files".
or use this search
index="_internal" source="*splunkd.log" earliest=-24h…"
Mike Langhorst replied to Dave P's discussion Splunk skipping input files & Order of precedence"a bit old on this but since I had an answer, hopefully it'll help you or someone else with this issue.
Correct, due to the first stanza, the second will not be used. What you can do is add an entry to specify this source type in props.conf…"
Dave P posted a discussionSplunk skipping input files & Order of precedence
Hi,So, I think I'm missing something obvious here. a 2 part question.1) I have one of several inputs defined as:monitor:///data/logs]disabled = falsehost_segment = 4index = defaultsourcetype = syslogThere are several directories under /data/logs/${DATE}. It appears that Splunk has "missed" them as files to index and add to the Splunk DB.Is there a way to kick Splunk to index files that it appears to have missed? Permissions, ownerships are correct... I'm stumped.2) Order of precedence.From…See More
Dave P replied to Joe Rizzo's discussion renaming searches, reports and dashboards"I haven't figured out how to rename them, but I did figure out that if you clone them and give them a new name, it's essentially the same thing. If it's not, I haven't discovered the difference yet.
-dave"
Michael Wilde replied to Dave P's discussion Optimizing searching over indexing"One more question... 32bit or 64bit?"
Dave P replied to Dave P's discussion Optimizing searching over indexing"The machine looks like this: 4 CPU cores, 2GB ram, 70GB disk, about 30GB used. When I profile the machine (top, sar, iostat, vmstat) I see that the splunkd processes (which appears to be the primary indexing agent) will consume up to 400% CPU (all…"
Michael Wilde replied to Dave P's discussion Optimizing searching over indexing"A couple of things come to mind...
What's the profile of your machine? CPU/Memory/Disk Space & Speed.
Are you finding that searches are slow? If so, what types of searches are you doing? How many are running simultaneously? Do you have…"
Dave P posted a discussionOptimizing searching over indexing
Hopefully, this will be first of many discussions I'll be part of. Found SN yesterday, very cool.I've got a single host handling both Splunk indexing and searching. I'd like to give searching priority over indexing. Ordinarily, being a UNIX hack I would nice +19 the splunk processes parent process.root 3492 11.7 12.7 341212 263652 ? SNl Nov09 335:00 splunkd -p 8080 restartroot 3493 0.0 0.0 17916 1728 ? SNs Nov09 0:28 splunkd -p 8080 restartroot 3543 0.9 2.2 187340 46368 ? SNl Nov09 26:25 python…See More
Profile Information
- Are you an existing splunk user?
- Licensed
- What do you do for your day job?
- UNIX geek
Comment Wall
- No comments yet!
© 2012 Created by Michael Wilde.
