Popular blog posts

Recent forum posts

Discussion Forum

Copying Apache log files to a database

Apache log files typically just sit around filling up your hard drive space, a vast array of text too unwieldy to search through.

A useful thing to do is to parse these files and copy them to a database. I've written a script that will do this using Rockets. It's not a web page-- it's a script that you should run from the command line (use sudo newlisp log-to-database.lsp) It will create a new database from scratch called SERVER-LOGS.db that you can then query and search using SQLite.

The script can be found here: https://github.com/newlisponrockets/newLISP-on-Rockets/blob/master/log-to-database.lsp

Note: I may be updating this later, as the database files seems quite large (only slightly smaller than the logs themselves)

Rocket Man on 2013-01-23 17:25:38.000

As far as queries, you can do all sorts of neat things.

For example, to display a list of all unique IPs (corresponds roughly to unique visitors) from the database, use:

(displayln (query "SELECT DISTINCT IP FROM Logs"))

It's that simple!

Views: 6092