Unix Logging
August 30, 2005 7:58 PM   Subscribe

Linuxfilter: I have a third-party app which writes occasional messages to its logfile. The problem is I dont know when these messages are logged. There is no timestamp - just the message. Is there any way I could force a timestamp? Maybe some trickery with 'tail-f' and 'date' and pipes into some new pseudo-logfile? Thanks.
posted by vacapinta to Computers & Internet (12 answers total)
Perhaps there's an option to turn on timestamps. I don't know any app which creates a logfile that doesn't include timestamps in said logfile.

You could tell cron to run a job every minute that:
-did a diff vs. last minute's file
-appended the diff output to a new file, with a timestamp
-copied the current file to last minute's file (so you'll have something to compare to in a minute)

That would give you one minute granularity with your timestamps. Kind of a hack, but eh. I'd read the manual/ask the app's authors how to turn on timestamps.
posted by jellicle at 8:20 PM on August 30, 2005

There's FileWatch, which would let you define an action. You'd want to echo a timestamp and tail -1 /log/yourfile into the pseudo-logfile. Cron it out to once a minute and you're good to go.

The truly hardcore would write a small daemon which redirects the log file into itself and outputs the correct syntax. Wouldn't be that hard, really.
posted by unixrat at 8:31 PM on August 30, 2005

After thinking about it for a second, I realize that my suggestion would flop if the output file updated more than once a minute.

If the output file can update at any time, you'll need a constantly running process which monitors it. That's not impossible to do, but does require a bit of advance knowledge.
posted by unixrat at 8:33 PM on August 30, 2005

A named pipe would be able to take the data as it is written and pass it to another program (such as a small Perl script) which can prepend a timestamp and write it to a new location. You can learn more about named pipes by reading the manpage for the 'mkfifo' command and with a healthy dose of Google.
posted by edd at 8:34 PM on August 30, 2005

Something like this should work:

tail -f logfile | perl -nle 'print localtime()." ".$_' > timestamp_logfile
posted by smackfu at 8:41 PM on August 30, 2005

Here's a dirty bit of awk-age that will do what you want, assuming you have something like what I run:

tail -f [your_log_file] | gawk '{"date" | getline cur_date} {close("date")} {print "[" cur_date "] " $0}'
posted by ChrisR at 8:52 PM on August 30, 2005

I second the named-pipe approach. Do yourself up a pipe to which the program should write its logfile, and have a Perl/Python/Awk/LISP/C/whatever program read from the pipe and echo it in with timestamps in place.
posted by Netzapper at 9:54 PM on August 30, 2005

An alternative to the (very good) suggestions about making a filter that adds the timestamp, see if you can enable syslog in the application. That will cause the messages to be sent through the syslog daemon which will add the timestamp for you.
posted by Rhomboid at 1:44 AM on August 31, 2005

You may well want to configure a specific syslog facility to log all messages from this application to a separate file. As an example, I use spamd to log messages about spam tarpitting to logfiles. The daemon that runs is spamd, and it logs to the daemon syslog facility. I have in /etc/syslog.conf this:

# For spamd, email relay tarpit
daemon.err;daemon.warn;daemon.info /var/log/spamd

This logs anything created by the spamd using the daemon logging facility to /var/log/spamd. I suspect you need something like this (assuming you're running the program mybinary):

# Your program
*.* /var/log/mybinary

Then restart syslogd with a HUP:

killall -HUP syslogd


kill -HUP `cat /var/run/syslogd.pid`

Failing that, an alternative logging tool such as multilog might be what you need. You can add any timestamp you want with that.
posted by gaby at 6:39 AM on August 31, 2005

Try installing syslog-ng. It works like the regular syslog daemon, but its configuration file lets you add other sources for log messages - like an individual file.

You can have syslog-ng watch this file, and pipe any line that appears there into your regular logfiles, or one made specifically for those messages. Syslog-ng adds its own timestamps along the way.
posted by tkolstee at 7:19 AM on August 31, 2005

There's a utility called dnotify that will let you run a command whenever a change to a directory is made. It uses Linux kernel 2.4.19+'s dnotify API, so you need to be running a kernel at least that recent.

The advantage over a solution requiring cron is that dnotify is triggered the instant the change in the filesystem is made whereas a cron solution needs to be run periodically (at best, every minute) to detect for a change.
posted by bachelor#3 at 7:47 AM on August 31, 2005

be aware that "tail -f " solutions have a possibility of failing if the filename changes (like for log rotating). Loof at the man pages for "tail -F".
posted by stovenator at 9:28 AM on August 31, 2005

« Older You're in good hands.   |   Current events timeline? Newer »
This thread is closed to new comments.