Here are some things I have done in the past to pull data out of log files. I find myself referring to this list from time to time to remember how I did things in the past.
Real Queries That May Need To Be Repeated ATT YIH upgrades in the month of 200709 $ listysftclog 200709* | grep att/v4 | perl -ne 'if(/(yihupg[^|]*)/){print $1."\n";}' | sort -r | uniq -c Scratch Work These searches are mostly captures in cronjobs and shell scripts. Using this page as a place to store some of them while working them out. Misc Searches $ cat ~/ysftc.log | grep att/v4 | perl -ne 'if(/beacon.choice~([^|]*)/){print $1."\n";}' | sed 's/&/\&/' | sort | uniq -c | sort -nr Get page views for att/v4 choice page $ cat ~/ysftc.log | grep att/v4 | grep choice.php | grep result~200 | wc -l Get uniq visits to choice page $ cat ~/ysftc.log | grep att/v4 | grep result~200 | cut -f10 -d\| | sort | uniq | wc -l Get all user agents This still needs some work, there is a ^D in there that needs to be removed $ cat /home/y/logs/ysftc/ysftc.log | grep att/v4 | perl -ne 'if(/choiceua([^|]*)/){print $1."\n";}' | sort | uniq -c | sort -nr Get Windows users $ cat /home/y/logs/ysftc/ysftc.log | grep att/v4 | perl -ne 'if(/choiceua([^|]*)/){print $1."\n";}' | grep Windows | wc -l Get Mac users $ cat /home/y/logs/ysftc/ysftc.log | grep att/v4 | perl -ne 'if(/choiceua([^|]*)/){print $1."\n";}' | grep Mac | wc -l Specific examples * Replace & in search results with & $ cat ~/ysftc.log | grep att/v4 | perl -ne 'if(/beacon.choice~([^|]*)/){print $1."\n";}' | sed 's/&/\&/' | sort | uniq -c | sort -nr 23 AT&T Yahoo! Online Protection 13 Parental Controls 11 AT&T Yahoo! Messenger 11 AT&T Yahoo! Browser 11 AT&T Toolbar 6 Connection Manager 2 AT&T Pop-Up Blocker 1 AT&T Security Suite SM