Tutorial: Grep-ing information from the Oscam log

[CapNCooK]

Hello,

Here are some hints to get more insight from your oscam log.

--

You need to have logging set to on:

oscam.conf
logfile = /tmp/oscam.log # <path_to>/oscam.log
maxlogsize = 99999 # <size_in_kilobytes> - note: *you must have enough space to store the size set here.. *


Now, lets get familiar with some basic linux commands:

cat /tmp/oscam.log
=> Outputs the full oscam.log

cat /tmp/oscam.log | more
=> Outputs the full oscam.log - paused at every page

--

tail /tmp/oscam.log
=> Outputs the last 10 lines from the log

tail -f /tmp/oscam.log
=> Keeps outputting a growing log, VERY useful to follow your log at runtime

tail -f /tmp/oscam.log | grep "found"
=> While Oscam is running and while your log is growing, this command outputs every line with "found" in it.
Very useful if you need to monitor "found ECM's"

--

Now you know these basic commands, these are the ones i use very often:

tail -f /tmp/oscam.log | grep <oscam_readername>
tail -f /tmp/oscam.log | grep <oscam_username>
=> Giving info about specific readers or users

tail -f /tmp/oscam.log | grep " c "
=> giving info about all user activities

tail -f /tmp/oscam.log | grep " p "
=> giving info about connection states of your readers

tail -f /tmp/oscam.log | grep "<CAID>"
=> giving info about specific card activities

Read the unfiltered log to invent more ideas

--

Little more advanced stuff..
You can even do some more advanced stuff.

If you use cache-ex, put the oscam log in debug mode 512 from the status screen, and you can use the following commands:

tail -f /tmp/oscam.log | grep "got"
=> giving an on-going list of new cached ecm's

tail -f /tmp/oscam.log | grep "duplicate"
=> giving an on-going list of blocked ecm's because of duplicate

--

With the values in the default log (debug level 0), you can gather some interesting statistics.
Here's a little script, parsing the oscam.log file, outputting some interesting stats.


Insert this into a new file on your oscam server (change the oscam.log path), and chmod it to 755 to use it:

#!/bin/sh

FILE=/var/log/oscam/oscam.log #CHANGE THIS!

echo "Top 10 of best readers:"
grep `date +%Y/%m/%d` $FILE | grep ' ms) by ' | awk '{print $11}' | sort | uniq -c | sort -nr | head -10
echo
echo "Top 10 users hammering on providers:"
grep `date +%Y/%m/%d` $FILE | grep ' ms) by ' | awk '{print $5$6}' | sed "s/(/ /" | cut -f 1 -d'/' | sort | uniq -c | sort -nr | head -10
echo
echo "Top 10 users by usage:"
grep `date +%Y/%m/%d` $FILE | grep ' ms) by ' | awk '{print $5}' | sort | uniq -c | sort -nr | head -10
echo
echo "Top 15 channels answered from cache:"
grep `date +%Y/%m/%d` $FILE | grep 'found' | grep 'cache' | awk '{print $12" "$13" "$14" "$15" "$16}' | sort | uniq -c | sort -nr | head -15
echo
echo "Top 15 channels that timed out:"
grep `date +%Y/%m/%d` $FILE | grep 'timeout' | grep 'by' | awk '{print $14" "$15" "$16}' | sort | uniq -c | sort -nr | head -15
echo
echo "Top 15 users that experienced time-outs (on channel):"
grep `date +%Y/%m/%d` $FILE | grep 'timeout' | grep 'by' | awk '{print $5" -- "$14" "$15" "$16}' | sort | uniq -c | sort -nr | head -15
echo

Hope this helps.. I will add some other stuff later :)

[PlatoonRTV]

WoW, nice piece of info! Especially script seems to be very useful.

EDIT:

echo "Top 15 channels answered from cache:, "Top 15 channels that timed out:","Top 15 users that experienced time-outs (on channel):"

I am getting 0 results for these.

[CapNCooK]

Could be so.. How big is the logfile ?
Is anything answered from cache ?

do a:

cat <logfile> | grep found | grep cache
cat <logfile> | grep timeout | grep by

to check

[PlatoonRTV]

Will report tomorrow. But i have cache3 and cache2 hits, so there should be some info

[CapNCooK]

Will report tomorrow. But i have cache3 and cache2 hits, so there should be some info

ok. Spaces in your reader- and/or user-names will cause problems with this script

[PlatoonRTV]

No, i am not using spaces for my readers and users. I only use letters, numbers and "_" sign.

I also tried these commands:

cat <logfile> | grep found | grep cache
cat <logfile> | grep timeout | grep by

First one gives me no results, second one seems to work fine.

[CapNCooK]

The first one should be:

cat <logfile> | grep found | grep by

This will show all the hits from cache...

Does that work ?

[mminkov]

Changing line: grep `date +%Y/%m/%d` $FILE | grep 'found' | grep 'cache' | awk '{print $12" "$13" "$14" "$15" "$16}' | sort | uniq -c | sort -nr | head -15
with: grep `date +%Y/%m/%d` $FILE | grep 'cache'|grep -v "n=-1." | grep -v "close" | awk '{print $12" "$13" "$14" "$15" "$16}' | sort | uniq -c | sort -nr | head -15

and script work beter.
I'm named this script `tdostat' - ToDayOscamStatistic and create another named `ostat'. After 1-st line (in 2-nd script), I insert code:

FILE=/var/log/oscam.log #CHANGE THIS!

if [ "$1" = "prev" ]
then
        FILE=/var/log/oscam.log-prev #CHANGE THIS!
fi

When execute script without argument - result is statistic for current log. When I execute script with argument `prev' - result is statistic from previous log file