Posts Tagged ‘Linux’

mod_security & mod_deflate

By Mark Davidson on February 24th, 2010

Recently with having mod_security running on this server I noticed some errors in the mod_security logs in relation to content encoding

[22/Feb/2010:20:41:06 +0000] [www.pablumfication.co.uk/sid#7f5543997918][rid#7f5543ef1228][/wp-admin/load-styles.php][2] Warning. Operator EQ matched 0 at GLOBAL. [file "/etc/apache2/conf.d/modsecurity/modsecurity_crs_30_http_policy.conf"] [line "120"] [id "960903"] [msg "ModSecurity does not support content encodings"] [severity "WARNING"]

I did a bit of research into the error and found a good post on Klaubert’s Blog – modsecurity vs content compression he suggests a mod_deflate config. He also mentions gmane Mail Archive Rule 960903 – content encodings post where the orignal config is from.

The config file seems to work perfectly but I just thought I would make this post to add a bit of a step by step to making this configuration file work under a standard Ubuntu 9.04 install running mod_security as it is likely that all the required modules will not be enabled by default.

First enable the following modules

sudo a2enmod ext_filter
sudo a2enmod headers
sudo a2enmod filter

Next using your favourite editor start editing /etc/apache2/mods-enabled/deflate.conf and comment out the existing content then add the following.

<IfModule mod_deflate.c>
 ExtFilterDefine nodeflate mode=output cmd=/bin/true \
 enableenv=SomeVarThatWillNeverBeSet

 SetOutputFilter DEFLATE

 # Netscape 4.x has some problems¦
 BrowserMatch ^Mozilla/4 gzip-only-text/html
 # Netscape 4.06-4.08 have some more problems
 BrowserMatch ^Mozilla/4\.0[678] no-gzip
 # IE is ok, but looked like Netscape, so we reset it
 BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

 SetEnvIfNoCase Accept-Encoding gzip force-gzip
 SetEnvIfNoCase TE gzip force-gzip

 RequestHeader unset Accept-Encoding
 RequestHeader unset TE

 # Skip images based on extension
 SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|zip|exe|swf|mp?eg|mp3|tgz|tar.gz|avi|ico|gz)$ no-gzip dont-vary

 # Make sure caching still works
 Header append Vary User-Agent env=!dont-vary

 <IfModule mod_security2.c>
 FilterDeclare modsec CONTENT_SET
 FilterProvider modsec modsecurity_out env=modsec-ignore !=1
 </IfModule>

 FilterDeclare compress CONTENT_SET
 FilterProvider compress deflate env=force-gzip =1
 #FilterProvider compress inflate Content-Type $image/jpeg
 #FilterProvider compress inflate Content-Type $image/gif
 #FilterProvider compress inflate Content-Type $image/png

 FilterProtocol compress "change=yes"
 <IfModule mod_security2.c>
 FilterChain modsec compress
 </IfModule>

 <IfModule !mod_security2.c>
 FilterChain compress
 </IfModule>
</IfModule>

Lastly restart apache

sudo apache2ctl graceful

and thats it done this should allow for mod_security & mod_deflate to run on the server together without erroring.

WordPress Auto Backup & Upgrade

By Mark Davidson on February 20th, 2010

As everyone I am sure knows to well upgrading wordpress can be a bit of a pain, so based on a couple of other scripts I found around the net I created this script for automatically backing up and then upgrading a wordpress install.

What it basically does is if you give it the full path of a wordpress install it will automatically reads the config file and get the DB details, then perform a backup of the db, then a backup of the wordpress files and then finally an upgrade of the wordpress install.

To run just use `bash wpUpgrader /var/www/wordpress/`

Download the Script Here

#!/bin/bash
# Linux WordPress Upgrader Script
#
# Mark Davidson | [email protected] | www.pablumfication.co.uk
#

function wpExtract {
	echo $(grep -o "define('$1', '\([^']*\)');" $WP/wp-config.php | cut -f 2 -d ' ' | awk '{ print substr($0, 2, length($0) - 4) }')
}

TEMP=/tmp

if [ $# = "1" ]; then
	WP=$1
	# need to either force full path input or create canconical version
	# Validate that this directory is valid and contains wordpress files 
	BACKUP_DIR=$PWD/backups/wp/$(date +%s)
	
	echo "Creating Backup Directory"
	mkdir -p $BACKUP_DIR
	echo "Backup Directroy Created " $BACKUP_DIR
	cd $BACKUP_DIR

	echo "Backing up wordpress database"
	result=`mysqldump -u $(wpExtract DB_USER) -p$(wpExtract DB_PASSWORD) --database $(wpExtract DB_NAME) 2>&1 > blog.sql` 
	if [ -n "$result" ]; then 
		echo $result
	else
		echo "Taring DB Backup"
		result=`tar -cf db.tar.gz blog.sql 2>&1`
	
		if [ -n "$result" ]; then
			echo "DB backup could not tared be created exiting"
		else
			echo "DB backup tared"
			rm blog.sql

			echo "Creating Backup of WordPress Files"
			tar -Pzcf blog.tar.gz $WP

			echo "Checking Backup Integrity"
			result=`tar -dPf blog.tar.gz $WP`
			
			if [ -n "$result" ]; then
				echo "Integrity check failed"
				echo $result
			else
				echo "Downloading Latest WordPress"
				wget -O $TEMP/latest.tar.gz http://wordpress.org/latest.tar.gz
				tar -zxf $TEMP/latest.tar.gz

				result=`tar -ztf $TEMP/latest.tar.gz | grep wordpress/index.php | wc -l` # Really basic chek that file is intact proberly a better way to do this
				
				if [ "$result" = 1 ]; then
					echo "File OK"

					echo "Extracting WordPress"
					result=`tar -C $TEMP -xf $TEMP/latest.tar.gz` #TODO: Validate extraction
					echo $result

					cd $WP
					result=`cp -avr $TEMP/wordpress/* .`

					echo $result

					rm -rf $TEMP/wordpress $TEMP/latest.tar.gz

					echo "New files copied vist http://yourdomain.com/wp-admin/upgrade.php to complete the upgrade"
				else 
					echo "File Corrupt or missing"
				fi
			fi
		fi
	fi
else 
	echo "Incorrect Number of Arguments"
fi

I based this script off a few others I suggested checking them out

Upgrade wordpress quickly in 3 easy steps from UNIX shell prompt
wordpress update script
WordPress Auto-Update Script For A Linux Server
WordPress Upgrade Script

VIM Python Code Folding

By Mark Davidson on February 11th, 2010

Today I have been working on my new project SysInfoRM and for this I am using Python as my language of choice and VIM as my editor. In an attempt to improve my experience with using VIM I have been checking out code folding after a bit of digging around I tracked down a good vimrc file. I found a good post on Amitu Blog detailing the vimrc and other associated details. Unfortunately the site was down when I visited and I had to rely on the version that is cached by Google.

So I thought I would take a few points from that blog post and expand on it a bit in case the copy in Google’s cache copy goes as well. I give full credit to the original author and would like to thank him for his time and effort on the original post.

Here are the important new lines for your .vimrc

" automatically save and restore folds
au BufWinLeave * mkview
au BufWinEnter * silent loadview

" this lets us put the marker in the file so that
" it can be shared across and stored in version control.
set foldmethod=marker
" this is for python, put
" # name for the folded text # {{{
" to begin marker and
" # }}}
" close to end it.
set commentstring=\ #\ %s
" default fold level, all open, set it 200 or something
" to make it all closed.
set foldlevel=0

" share clipboard with windows clipboard
set clipboard+=unnamed

" set viminfo='100,f1
" minibufexplorer settings:j
let g:miniBufExplMapWindowNavArrows = 1
let g:miniBufExplMapCTabSwitchWindows = 1

These are the keyboard command for using the folds.

  • zf create the fold, useful for manual and marker methods. Select any piece of text, [press v or shift-v, then use arrow keys], and then press zf. It will place the markers around the fold for you in marker mode; in case of manual, it will store fold location in memory. Remember f by saying this command “forms” the fold, or just remember fold :-)
  • zc close the fold at the cursor.
  • zo open the fold at the cursor.
  • zr increment the fold level by one, so if all classes are folded, they will opened, but function definitions will be kept folded.
  • zm reverse of the above, if one or more function folds are open, they will be closed, but classes will be kept open.
  • zR open all folds.
  • zM close all folds.
  • zj and zk can be used to jump from one fold to another.

Here is an example of what should expect. This is some example Python code.

from xml.parsers.xmlproc import xmlproc
from xml.parsers.xmlproc import xmlval
from xml.parsers.xmlproc import xmldtd

# XML Structure Checking # {{{
class MyApp(xmlproc.Application):
  def handle_start_tag(self,name,attrs):
    pass
  def handle_end_tag(self,name):
    pass
  def handle_data(self,data,start,end):
    pass
  def handle_comment(self,data):
    pass
# }}}

Placing the cursor at and point after line 5 then using zc, would result in.

from xml.parsers.xmlproc import xmlproc
from xml.parsers.xmlproc import xmlval
from xml.parsers.xmlproc import xmldtd

+-- 11 lines: XML Structure Checking-----------------------------------------------

That is all for now I plan to expand a bit on these vimrc modifications tomorrow and will then publish my entire vimrc which is becoming quite extensive.