The dojo of Splunk. Learn, share, teach, mentor.
Videos
Latest Activity
Michael Wilde replied to Nikita's discussion Count failures and success via transaction"How are these transactions linked together... by a field called "ID"? If so.. just build them with the field ID, and then use one of the MV commands to extract a field with success or failure in it. Paste some samples and…"
nicholas Lehman's Discussions
Search returns weird #'s
Ok, here's the deal. Still working on the AS/400 Splunk issue. I am currently focusing on creating event types and custom searches for particular security events from the AS/400. Based on my…Continue
Started Mar 23, 2010
Timestamping is the bane of my existance
10 Replies
I'm working on a quick fix for AS/400 logging and the ability to cleanly report it. I've got all the fields mapped out, but the timestamping gets retarded. I've tried feeding the logfile to splunk…Continue
Started this discussion. Last reply by nicholas Lehman Mar 8, 2010.
nicholas Lehman's Page
Gifts Received
nicholas Lehman has not received any gifts yet
Latest Activity
nicholas Lehman posted a discussionSearch returns weird #'s
Ok, here's the deal. Still working on the AS/400 Splunk issue. I am currently focusing on creating event types and custom searches for particular security events from the AS/400. Based on my understanding of IBM's documentation (leaves much to be desired, btw) the security related events are made up of two sections of an event. The Journal Code and Event Type. The Journal Code for security should always be a T, and the event type will be a two letter code that corresponds to different security…See More
nicholas Lehman replied to nicholas Lehman's discussion Timestamping is the bane of my existance"Well, if this is a common occurrence, I may want to look into developing an app for AS/400. In the meantime, I can post the script I made for my particular application. Simple bash (cause perl confuses me).
#!/bin/bash
cd ~/
echo accessing…"
Michael Wilde replied to nicholas Lehman's discussion Timestamping is the bane of my existance"If your script is "post-able".. that'd be great... the "AS/400 question" comes up often and users would like to benefit from your experience."
nicholas Lehman replied to nicholas Lehman's discussion Timestamping is the bane of my existance"Well, this is a down and dirty compliance stop gap. The log journals get spat out to text file every night, and then a cron job runs a script to pull them off the as/400 via sftp along with an MD5 sum file. The script then performs an md5sum on the…"
Michael Wilde replied to nicholas Lehman's discussion Timestamping is the bane of my existance"Question for you:
How'd you get the logs from the AS/400. Arent they in EBCDIC, if so did you convert them? Whats your strategy for AS/400. Please share. (might even be blog post worthy!)"
nicholas Lehman replied to nicholas Lehman's discussion Timestamping is the bane of my existance"haha, I was just about to come back and let you know I answered my own question. Thanks for all the help Michael!"
Michael Wilde replied to nicholas Lehman's discussion Timestamping is the bane of my existance"You are correct. Monitor a directory will work just fine. I user a single file to answer your challenge. Glad it worked for you!"
nicholas Lehman replied to nicholas Lehman's discussion Timestamping is the bane of my existance"Awesome, it worked! One question though, for the monitor option in the inputs.conf, can I just leave the end of the path open ended to look in that directory for any new files? I see you put a file at the end, and am wondering if I'm a bit…"
Michael Wilde replied to nicholas Lehman's discussion Timestamping is the bane of my existance"This will work.
First, create a props.conf in an appropriate directory (system level or app level), I'll make mine in $SPLUNK_HOME/etc/system/local/props.conf
Add this to it:
#my sourcetype will be called "as400"
[as400]
#I want…"
nicholas Lehman replied to nicholas Lehman's discussion Timestamping is the bane of my existance"the information preceeding the timestamp will always be different. The beginning of the event log contains record length, sequence number, journal code and entry type. Even the information immediately before and after the timestamp will be different…"
Michael Wilde replied to nicholas Lehman's discussion Timestamping is the bane of my existance"Nicholas.. can we establish a pattern on what precedes the timestamp... like will it always be some number of digits, followed by a TPW, followed by your date pattern?
If so.. this should be easy.."
nicholas Lehman posted a discussionTimestamping is the bane of my existance
I'm working on a quick fix for AS/400 logging and the ability to cleanly report it. I've got all the fields mapped out, but the timestamping gets retarded. I've tried feeding the logfile to splunk for timestamp recognition. Even tried a bit of regex (which I most likely biffed). Nothin'. Pulling my hair out here guys. Below is the first part of a log event Time stamp is "021110070911", or "MMDDYYhhmmss.Any ideas on getting proper timestamp information out of…See More
Michael Wilde left a comment for nicholas Lehman"Nicholas..
Thanks for signing up. I set up this site so we can freely share, discuss, ask questions, post videos.. whatever. Ask hard questions, and we'll try to get them answered.
Milwaukee? I grew up there (West Allis to be specific).…"
Profile Information
- Are you an existing splunk user?
- Free
- What do you do for your day job?
- Information Security
Comment Wall (1 comment)
- At 8:16pm on January 7, 2010, Michael Wilde said…
- Nicholas..
Thanks for signing up. I set up this site so we can freely share, discuss, ask questions, post videos.. whatever. Ask hard questions, and we'll try to get them answered.
Milwaukee? I grew up there (West Allis to be specific). Austin, TX is my home now.
Thanks
Michael Wilde
Splunk Ninja
© 2012 Created by Michael Wilde.
