Building Mellanox OFED 1.4 for Rocks 5.3

Here are my notes from building Mellanox OFED 1.4 on a Rocks 5.3 x86_64 cluster utilizing CentOS 5.4 and kernel 2.6.18-128-7.1:

1. Download the ISO file MLNX_OFED_LINUX-1.4-rhel5.3.iso from this page

The MLNX_OFED-1.4 comes with modules for kernel 2.6.18-128, we are using 2.6.18-128-7.1, so we need to build new modules.

2. Mount the ISO and copy the contents to a scratch work area

# mount -t iso9660 -o loop /root/MLNX_OFED_LINUX-1.4-rhel5.3.iso /mnt/cdrom 
# mkdir /root/MLNX_OFED-1.4
# cp -r /mnt/cdrom/* /root/MLNX_OFED_LINUX-1.4/
# umount /mnt/cdrom

3. Edit the script so that it will work with CentOS (our centos-release says 5.4, we are still running a 5.3 kernel), this is the script that will build a new ISO file

# cd /root/MLNX_OFED-1.4

TabularUnifieddocs/mlnx_add_kernel_support.sh
Index: docs/mlnx_add_kernel_support.sh
===================================================================
--- docs/mlnx_add_kernel_support.sh.orig 2009-12-17 15:51:46.000000000 -0600
+++ docs/mlnx_add_kernel_support.sh 2009-12-17 15:52:00.000000000 -0600
@@ -279,7 +279,7 @@
         redhat-release-5Server-5.2.0.4)
         distro="rhel5.2"
         ;;
-        redhat-release-5Server-5.3.0.3)
+        redhat-release-5Server-5.3.0.3 | centos-release-5-4.el5.centos.1 )
         distro="rhel5.3"
         ;;
         sles-release-10-15.2)

4. Install some dependencies

# yum -y install libtool tcl-devel libstdc++-devel mkisofs gcc-c++

5. Uninstall some RPM files that will fail to uninstall during the ISO build

/bin/rpm --nodeps -e --allmatches openmpi-libs-1.3.2-2.el5 \
 openmpi-devel-1.3.2-2.el5 rocks-openmpi-1.3.2-1 openmpi-libs-1.3.2-2.el5 \
 openmpi-devel-1.3.2-2.el5 openmpi-1.3.2-2.el5 openmpi-1.3.2-2.el5 \
 openmpi-gnu-1.3.3-1.el5.uabeng

6. Build the new ISO file

# cd /root/MLNX_OFED_LINUX-1.4
# ./docs/mlnx_add_kernel_support.sh -i /root/MLNX_OFED_LINUX-1.4-rhel5.3.iso

Note: This program will create MLNX_OFED_LINUX ISO for rhel5.3 under /tmp directory.
      All Mellanox, OEM, OFED, or Distribution IB packages will be removed.
Do you want to continue?[y/N]:y
Building OFED RPMs...
Removing OFED RPMs...
Running mkisofs...
Created /tmp/MLNX_OFED_LINUX-1.4-rhel5.3.iso

# mv /tmp/MLNX_OFED_LINUX-1.4-rhel5.3.iso /share/apps/mellanox/MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1.iso

7. Copy the new files from the iso to the NFS share

# mount -t iso9660 -o loop /share/apps/mellanox/MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1.iso /mnt/cdrom
# cp -r /mnt/cdrom /share/apps/mellanox/MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1

# cd /share/apps/mellanox
# find ./MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1 -name kernel-* | grep x86
./MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1/x86_64/kernel-ib-1.4-2.6.18_128.7.1.el5.x86_64.rpm
./MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1/x86_64/kernel-ib-1.4-2.6.18_128.el5.x86_64.rpm
./MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1/x86_64/kernel-ib-devel-1.4-2.6.18_128.el5.x86_64.rpm
./MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1/x86_64/kernel-ib-devel-1.4-2.6.18_128.7.1.el5.x86_64.rpm

8. Test the installer on the compute node

# cd /share/apps/mellanox/MLNX_OFED_LINUX-1.4-rhel5.3-kernel-2.6.18_128.7.1
# ./mlnxofedinstall --hpc

This program will install the MLNX_OFED_LINUX package on your machine.
Note that all other Mellanox, OEM, OFED, or Distribution IB packages will be removed. 
Do you want to continue?[y/N]:y

Uninstalling the previous version of OFED 

Starting MLNX_OFED_LINUX-1.4 installation ... 

Installing mpi-selector RPM 
Preparing...                ########################################### [100%]
   1:mpi-selector           ########################################### [100%]
Installing kernel-ib RPM 
Preparing...                ########################################### [100%]
   1:kernel-ib              ########################################### [100%]
Installing ib-bonding RPM 
Preparing...                ########################################### [100%]
   1:ib-bonding             ########################################### [100%]
Installing mft RPM 
Preparing...                ########################################### [100%]
   1:mft                    ########################################### [100%]
Install user level RPMs: 
Preparing...                ########################################### [100%]
   1:libibverbs             ########################################### [  2%]
   2:libibcommon            ########################################### [  4%]
   3:libibumad              ########################################### [  6%]
   4:opensm-libs            ########################################### [  8%]
   5:librdmacm              ########################################### [ 10%]
   6:openmpi_intel          ########################################### [ 12%]
   7:libibmad               ########################################### [ 14%]
   8:infiniband-diags       ########################################### [ 16%]
   9:openmpi_gcc            ########################################### [ 18%]
  10:mpitests_openmpi_gcc   ########################################### [ 20%]
  11:mpitests_openmpi_pgi   ########################################### [ 22%]
  12:mpitests_openmpi_intel ########################################### [ 24%]
  13:qperf                  ########################################### [ 26%]
  14:perftest               ########################################### [ 28%]
  15:ibutils                ########################################### [ 30%]
  16:libmthca               ########################################### [ 32%]
  17:libmlx4                ########################################### [ 34%]
  18:openmpi_pgi            ########################################### [ 36%]
  19:mstflint               ########################################### [ 38%]
  20:mlnxofed-docs          ########################################### [ 40%]
  21:ofed-scripts           ########################################### [ 42%]
  22:libibverbs             ########################################### [ 44%]
  23:libibcommon            ########################################### [ 46%]
  24:libibumad              ########################################### [ 48%]
  25:mvapich_intel          ########################################### [ 50%]
  26:opensm-libs            ########################################### [ 52%]
  27:librdmacm              ########################################### [ 54%]
  28:libibcommon-devel      ########################################### [ 56%]
  29:libibumad-devel        ########################################### [ 58%]
  30:libibverbs-devel       ########################################### [ 60%]
  31:librdmacm-utils        ########################################### [ 62%]
  32:opensm                 ########################################### [ 64%]
  33:mvapich_gcc            ########################################### [ 66%]
  34:mpitests_mvapich_gcc   ########################################### [ 68%]
  35:mpitests_mvapich_pgi   ########################################### [ 70%]
  36:mpitests_mvapich_intel ########################################### [ 72%]
  37:mvapich_pgi            ########################################### [ 74%]
  38:libibverbs-utils       ########################################### [ 76%]
  39:librdmacm-devel        ########################################### [ 78%]
  40:librdmacm-devel        ########################################### [ 80%]
  41:opensm-devel           ########################################### [ 82%]
  42:opensm-devel           ########################################### [ 84%]
  43:libibumad-devel        ########################################### [ 86%]
  44:libibcommon-devel      ########################################### [ 88%]
  45:libibverbs-devel       ########################################### [ 90%]
  46:libibmad               ########################################### [ 92%]
  47:libmthca               ########################################### [ 94%]
  48:libmlx4                ########################################### [ 96%]
  49:libibmad-devel         ########################################### [ 98%]
  50:libibmad-devel         ########################################### [100%]
Device (15b3:673c):
        0c:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev a0)
        Link Width: 8x
        Link Speed: 2.5Gb/s


Installation finished successfully. 

The firmware version 2.6.0 is up to date. 
Note: To force firmware update use '--force-fw-update' flag.
Configuring /etc/security/limits.conf. 
warning: /etc/infiniband/openib.conf saved as /etc/infiniband/openib.conf.rpmsave

How To Build the ATI Fglrx kernel modules for a new CentOS 5 kernel

RHEL5 / CentOS5 kernel update time can be troublesome with regards to workstation video cards.

Especially if you are dealing with 10's of 100's of workstations, a simple 'yum update' can result in a large number of support calls stating, "I rebooted my workstation, and now it reports that X can't start!"

Workstations with nVidia based cards can use the kmod RPMs provided by the ELRepo repository (is that redundant?).

Unfortunately, there aren't kmod RPMs for ATI Radeon cards (fglrx). Until I get around to rolling my own kmod RPM, here's the solution I use (I originally posted this script on this blog post):

1. Many kernels ago, I had built an RPM using the ati-driver installation package and installed the RPM

$ sh ati-driver-installer-9-3-x86.x86_64.run --buildpkg RedHat/RHEL5_64a
$ sudo rpm -Uvh fglrx64_7_1_0-8.593-1.x86_64.rpm

2. Create a script that will build a new kernel module based on the currently loaded kernel: /root/fglrx_build_kernel_module.sh

#!/bin/sh
# Builds updated kernel module (based on currently loaded kernel)
# 
# This driver was installed via RPM, built using:
# sh ati-driver-installer-9-3-x86.x86_64.run --buildpkg RedHat/RHEL5_64a

# fix the source code to work with CVE-2010-3081
/usr/bin/perl -pi -e 's/return compat_alloc_user_space/return arch_compat_alloc_user_space/g;' /lib/modules/fglrx/build_mod/kcl_ioctl.c

cd /lib/modules/fglrx/build_mod
./make.sh

cd /lib/modules/fglrx
./make_install.sh

echo "To complete the process you must 'init 3' and 'init 5'"
echo "lsmod | grep fglrx"

3. Update the kernel followed by a reboot and you'll notice that X fails to start and that the fglrx module is not loaded. Enter runlevel 3, and run the script

# lsmod | grep fglrx

# init 3
# sh ./fglrx_build_kernel_module.sh

AMD kernel module generator version 2.1
doing Makefile based build for kernel 2.6.x and higher
rm -rf *.c *.h *.o *.ko *.GCC* .??* *.symvers
make -C /lib/modules/2.6.18-194.8.1.el5/build SUBDIRS=/lib/modules/fglrx/build_mod/2.6.x modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-194.8.1.el5-x86_64'
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/firegl_public.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_acpi.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_agp.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_debug.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_ioctl.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_io.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_pci.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_str.o
  CC [M]  /lib/modules/fglrx/build_mod/2.6.x/kcl_wait.o
  LD [M]  /lib/modules/fglrx/build_mod/2.6.x/fglrx.o
  Building modules, stage 2.
  MODPOST
  CC      /lib/modules/fglrx/build_mod/2.6.x/fglrx.mod.o
  LD [M]  /lib/modules/fglrx/build_mod/2.6.x/fglrx.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-194.8.1.el5-x86_64'
build succeeded with return value 0
duplicating results into driver repository...
done.
You must change your working directory to /lib/modules/fglrx
and then call ./make_install.sh in order to install the built module.
- recreating module dependency list
- trying a sample load of the kernel modules
done.
To complete the process you must 'init 3' and 'init 5'

4. Once the script finishes, enter runlevel 5 and verify that the fglrx module is loaded

# init 5

# lsmod | grep fglrx
fglrx                2389736  3 

To Do: Rewrite the script to build the kernel module for whichever kernel you specify instead of only the currently loaded kernel. nVidia provides support for this, for example to build the module for the latest EL5 kernel:

# sh NVIDIA-Linux-x86_64-190.42-pkg2.run -a -K -k 2.6.18-194.8.1.el5

VMware Announces the Release of vSphere 4.1

The release over can be found on this page.

The page is mostly sales jargon, but does contain one interesting tidbit that ESX will not be supported after this release!

"For planning purposes, it is important to know that vSphere 4.1 is the last vSphere release to support both ESX and ESXi hypervisor architectures; all future releases of vSphere will only support the ESXi architecture"

Also, some exciting news for Essentials Plus users:

"VMware vMotion™ is now included in the Essentials Plus and Essentials for Retail & Branch Offices kits, as well as in vSphere Standard edition."

Installing an Subversion 1.6 and Trac 0.12 server on CentOS 5.5

These are some notes that I took while setting up a new server to run Subversion 1.6.11 and Trac 0.12 on CentOS 5.5

I'm not claiming this is the perfect setup (it's probably far from that). I had a short window in which to get a dead SVN physical server back online as a VMware ESXi virtual machine.

The old server ran a very old install of Gentoo and SVN 1.1.3 along with Trac 0.9.3.

This server is also only access via a local network.

As always, feedback is welcome.

If I had the time, I would have preferred to build RPMs for each of the installed components...

Kickstart

# Kickstart file automatically generated by anaconda.
# boot: linux ks=http://<kickstart-source>/kickstart/svnsrv-centos5.cfg
#       ip=10.0.1.10 netmask=255.255.255.0 gateway=10.0.1.1 nameserver=10.0.1.1
# Subverions repository server

install
#cdrom
url --url http://mirror.mydom/distro-trees/centos-5-i386
lang en_US.UTF-8
langsupport --default=en_US.UTF-8 en_US.UTF-8
keyboard us
text

network --device eth0 --bootproto static --ip 10.0.1.10 --netmask 255.255.255.0 --gateway 10.0.1.1 --nameserver 10.0.1.1 --hostname svnsrv.mydom
rootpw --iscrypted xxxxxxxxxxxxxxxxxxxx
reboot
# Ports to open on the firewall, tcp port 5666 is for nagios client
firewall --enabled --port=22:tcp --port=443:tcp
selinux --enforcing
authconfig --enableshadow --enablemd5 --enableldapauth --enableldaptls --ldapserver ldap.mydom --ldapbasedn dc=my,dc=dom
timezone --utc America/Chicago
# For Virtual Machines, see this VMware KB to determine what grub setting (if any) is needed
# to keep the time in sync
# http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427
#   32bit: --append="clock=pit"
#   32bit RHEL 5.4:     --append="divider=10"
#         RHEL 5.3,2,1: --append="clocksource=acpi_pm divider=10"
#   64bit RHEL 5.4:     --append="divider=10"
#         RHEL 5.3,2,1: --append="notsc divider=10"
bootloader --location=mbr --md5pass=xxxxxxxxxxxxxxxxxxxxxx --append="rhgb quiet divider=10"
clearpart --all --drives=sda --initlabel
part /boot --fstype ext3 --size=256 --ondisk=sda
part pv.2 --size=0 --grow --ondisk=sda
volgroup vg_svnsrv --pesize=32768 pv.2
part pv.3 --size=0 --grow --ondisk=sdb
volgroup vg_backup --pesize=32768 pv.3
logvol swap  --fstype swap --name=lv_swap  --vgname=vg_lmgr --recommended
logvol /var --fstype ext4 --name=lv_var --vgname=vg_svnsrv --size=4096
logvol /tmp --fstype ext3 --name=lv_tmp --vgname=vg_svnsrv --size=1024
logvol / --fstype ext4 --name=lv_root --vgname=vg_svnsrv --size=20480
logvol /home --fstype ext4 --name=lv_home --vgname=vg_svnsrv --size=1024 --grow
logvol /backup --fstype ext4 --name=lv_backup --vgname=vg_backup --size=1024 --grow

%packages --resolvedeps --nobase
# Minimal components to install
NetworkManager
audit
bzip2
crontabs
dhclient
logrotate
mailx
man
ntp
openssh
openssh-clients
openssh-server
pam_passwdqc
postfix
psacct
screen
sudo
tcpdump
telnet
wget
which
yum
# Additional tools
@development-libs
kernel-devel
rsync
gcc
vim-enhanced
make
autoconf
libtool
httpd
httpd-devel
mod_ssl
db4
db4-devel
db4-utils
db4-tcl
tcsh
# Needed to allow ldap authentication for BlazerIds
nss_ldap
nscd
# remove these
-rwho
-rusers
-sysreport
-system-config-httpd
-system-config-nfs
-system-config-samba

%post
###########################################################
### Create script to configure the server at first boot ###
###########################################################
/sbin/chkconfig ntpd on
/sbin/service ntpd start

########################################
# Enable/Disable Services
/sbin/chkconfig smartd on
/sbin/chkconfig isdn off
/sbin/chkconfig pcmcia off
/sbin/chkconfig openibd off

########################################
# Add user accounts (no passwords, let ldapauth handle that
/usr/sbin/authconfig --kickstart --enablelocauthorize
/usr/sbin/groupadd -g 10000 flakrat
/usr/sbin/useradd -u 10000 -g 10000 -c "FlakRat's Rat House" flakrat
/usr/sbin/usermod -a -G wheel flakrat
/bin/sed -i 's/^# \(%wheel\tALL=(ALL)\tALL\)$/\1/' /etc/sudoers

# Perform a full update
/bin/rm -rf /var/cache/yum/*
/usr/bin/yum clean all
rm -rf /etc/yum.repos.d/*

/usr/bin/yum -y update 2>&1 > /root/yum-update.log

##########################################
#Return rc.local back to the original copy so we don't rerun this each reboot
/bin/mv /etc/rc.d/rc.local /etc/rc.d/rc.local.firstboot
/bin/mv /etc/rc.d/rc.local.sav /etc/rc.d/rc.local

# Reboot the system to pick up all the updates
/sbin/shutdown -r now

EOF3
# make configuration script executable
chmod +x /root/postinstall-config.sh

#####################################################
# save a copy of rc.local
cp /etc/rc.d/rc.local /etc/rc.d/rc.local.sav

# add postinstall-config.sh to rc.local
cat >> /etc/rc.d/rc.local <<EOF
/root/postinstall-config.sh
EOF
#

Once the kickstart is complete, install the VMware tools and enable puppet client.

After which, add the SVN admin users to the svnadmin group and provide access to the svn home

$ sudo /usr/sbin/usermod -a -G svnadmin flakrat
$ sudo /usr/sbin/usermod -a -G svnadmin otheruser
$ sudo /usr/sbin/usermod -a -G svnadmin apache

$ sudo chmod g+rwx /home/svn
$ sudo chmod g+s /home/svn

Http Server

Self Signed Certificate

1. Create a working directory to create the certificates (the symlinks are simply helpful reminders for where the cert and key will be stored)
   

   
   $ sudo su -
   # mkdir /root/certificates
   # cd /root/certificates
   # ln -s /etc/pki/tls/private
   # ln -s /etc/pki/tls/certs
   
   

2. Generate the new SSL private key
   

   
   # openssl genrsa 4096 > svnsrv.mydom.key
   
   Generating RSA private key, 4096 bit long modulus
   .......................................................................................................................................................................++
   .................................++
   e is 65537 (0x10001)
   
   

3. Generate the self signed certificate (make sure that the common name is correct, otherwise newer browsers will reject the cert!)
   

   
   # openssl req -new -key svnsrv.mydom.key -x509 -days 3650 -out svnsrv.mydom.crt
   
   You are about to be asked to enter information that will be incorporated
   into your certificate request.
   What you are about to enter is what is called a Distinguished Name or a DN.
   There are quite a few fields but you can leave some blank
   For some fields there will be a default value,
   If you enter '.', the field will be left blank.
   -----
   Country Name (2 letter code) [GB]:US
   State or Province Name (full name) [Berkshire]:Texas
   Locality Name (eg, city) [Newbury]:College Station
   Organization Name (eg, company) [My Company Ltd]:Some Company
   Organizational Unit Name (eg, section) []:Some Department
   Common Name (eg, your name or your server's hostname) []:svnsrv.mydom
   Email Address []:flakrat@mydom
   
   

4. Copy the cert and key to the correct location
   

   
   # chmod 600 svnsrv.*
   # cp svnsrv.mydom.crt /etc/pki/tls/certs/
   
   # cp svnsrv.mydom.key /etc/pki/tls/private/
   
   

5. Edit ssl.conf to use the new cert and key
   

   
   # vim /etc/httpd/conf.d/ssl.conf 
   
   SSLEngine on
   
   #SSLCertificateFile /etc/pki/tls/certs/localhost.crt
   SSLCertificateFile /etc/pki/tls/certs/svnsrv.mydom.crt
   
   #SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
   SSLCertificateKeyFile /etc/pki/tls/private/svnsrv.mydom.key
   
   

6. Restart apache

Python

1. Download the latest Python software (the OS installed version is 2.4.x, we'll use 2.6.5)
   

   
   $ sudo mkdir /opt/python
   $ sudo chown flakrat:flakrat /opt/python
   
   $ mkdir ~/software/python
   $ cd ~/software/python
   $ wget http://www.python.org/ftp/python/2.6.5/Python-2.6.5.tar.bz2
   $ wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg
   $ tar -jxf Python-2.6.5.tar.bz2
   
   

2. Compile Python and install it
   

   
   $ cd Python-2.6.5
   $ make clean
   $ ./configure --prefix=/opt/python
   $ make
   
   Failed to find the necessary bits to build these modules:
   _tkinter           bsddb185           sunaudiodev     
   To find the necessary bits, look in setup.py in detect_modules() for the module's name.
   
   $ make install
   
   

3. Install setuptools
   

   
   $ alias python="/opt/python/bin/python2.6"
   $ export PATH=/opt/python/bin:$PATH
   
   $ python -V
   Python 2.6.5
   
   $ cd ~/software/python
   $ sh setuptools-0.6c11-py2.6.egg
   
   Processing setuptools-0.6c11-py2.6.egg
   Copying setuptools-0.6c11-py2.6.egg to /opt/python/lib/python2.6/site-packages
   Adding setuptools 0.6c11 to easy-install.pth file
   Installing easy_install script to /opt/python/bin
   Installing easy_install-2.6 script to /opt/python/bin
   
   Installed /opt/python/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg
   Processing dependencies for setuptools==0.6c11
   Finished processing dependencies for setuptools==0.6c11
   
   

4. Install a new mod_python
   

   
   $ alias python="/opt/python/bin/python2.6"
   
   $ cd ~/software/python
   $ wget http://mirrors.ecvps.com/apache/httpd/modpython/mod_python-3.3.1.tgz
   $ tar -zxf mod_python-3.3.1.tgz
   $ cd mod_python-3.3.1
   $ ./configure --with-apxs=/usr/sbin/apxs
   $ make
   
   make[1]: Entering directory `/home/flakrat/software/python/mod_python-3.3.1'
   make[2]: Entering directory `/home/flakrat/software/python/mod_python-3.3.1/src'
   make[2]: Leaving directory `/home/flakrat/software/python/mod_python-3.3.1/src'
   make[2]: Entering directory `/home/flakrat/software/python/mod_python-3.3.1/dist'
   make[2]: `build' is up to date.
   make[2]: Leaving directory `/home/flakrat/software/python/mod_python-3.3.1/dist'
   
   Now su and make install
     (or, if you only want to perform a partial install,
      you can use make install_dso and make install_py_lib)
   
   $ make install
   
   Performing DSO installation.
   
   /usr/bin/install -c -d /usr/lib/httpd/modules
   /usr/bin/install: cannot change permissions of `/usr/lib/httpd/modules': Operation not permitted
   make[1]: *** [install_dso] Error 1
   make[1]: Leaving directory `/home/flakrat/software/python/mod_python-3.3.1'
   make[1]: Entering directory `/home/flakrat/software/python/mod_python-3.3.1'
   cd dist && make install_py_lib
   make[2]: Entering directory `/home/flakrat/software/python/mod_python-3.3.1/dist'
   make[3]: Entering directory `/home/flakrat/software/python/mod_python-3.3.1/src'
   make[3]: `psp_parser.c' is up to date.
   make[3]: Leaving directory `/home/flakrat/software/python/mod_python-3.3.1/src'
   if test -z "" ; then \
     /opt/python/bin/python setup.py install --optimize 2 --force ; \
    else \
     /opt/python/bin/python setup.py install --optimize 2 --force --root  ; \
    fi
   running install
   running build
   running build_py
   running build_ext
   running install_lib
   creating /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/util.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/cache.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/psp.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/cgihandler.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/_psp.so -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/publisher.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/importer.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/__init__.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/testhandler.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/python22.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/Cookie.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/Session.py -> /opt/python/lib/python2.6/site-packages/mod_python
   copying build/lib.linux-i686-2.6/mod_python/apache.py -> /opt/python/lib/python2.6/site-packages/mod_python
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/util.py to util.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/cache.py to cache.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/psp.py to psp.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/cgihandler.py to cgihandler.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/publisher.py to publisher.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/importer.py to importer.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/__init__.py to __init__.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/testhandler.py to testhandler.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/python22.py to python22.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/Cookie.py to Cookie.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/Session.py to Session.pyc
   byte-compiling /opt/python/lib/python2.6/site-packages/mod_python/apache.py to apache.pyc
   writing byte-compilation script '/tmp/tmpgowmQp.py'
   /opt/python/bin/python -OO /tmp/tmpgowmQp.py
   removing /tmp/tmpgowmQp.py
   running install_egg_info
   Writing /opt/python/lib/python2.6/site-packages/mod_python-3.3.1-py2.6.egg-info
   make[2]: Leaving directory `/home/flakrat/software/python/mod_python-3.3.1/dist'
   make[1]: Leaving directory `/home/flakrat/software/python/mod_python-3.3.1'
   
   

5. Now copy the new mod_python.so library to the httpd modules folder
   

   
   $ sudo cp ~/software/python/mod_python-3.3.1/src/.libs/mod_python.so \
    /etc/httpd/modules/mod_python26.so
   
   

6. edit python.conf to point to the new library
   

   
   $ sudo vim /etc/httpd/conf.d/python.conf
   
   

   
   #
   # Mod_python is a module that embeds the Python language interpreter
   # within the server, allowing Apache handlers to be written in Python.
   #
   
   # 20100614 - MJH replacing system mod_python with recompiled version for python 2.6
   #LoadModule python_module modules/mod_python.so
   LoadModule python_module modules/mod_python26.so
   
   # Override type-map handler for /var/www/manual
   <Directory "/var/www/manual/mod/mod_python">
    <Files *.html>
     SetHandler default-handler
    </Files>
   </Directory>
   
   # This will cause files beneath /var/www/html with the extension .spam
   # to be handled by the Python script /var/www/html/eggs.py
   #
   #<Directory /var/www/html>
   #    AddHandler python-program .spam
   #    PythonHandler eggs
   #</Directory>
   
   # This will cause all requests to the /python heirachy of your
   # webserver to be handled by the python script /path/to/myhandler.py
   #
   #<Location /python>
   #    SetHandler python-program
   #    PythonPath "sys.path + ['/path/to']"
   #    PythonHandler myhandler
   #</Location>
   
   # This will cause all requests to the /python hierachy of your
   # webserver to be handled by mod_python's Publisher handler
   # (see http://localhost/manual/mod/mod_python/hand-pub.html)
   #
   #<Location /python>
   #    SetHandler python-program
   #    PythonHandler mod_python.publisher
   #</Location>
   
   # This will cause the output of all requests to files beneath
   # /var/www/html with the extension .flt to be filtered through
   # the Python script /var/www/html/filter.py
   #
   #<Directory /var/www/html>
   #    PythonOutputFilter filter MYFILTER
   #    AddOutputFilter MYFILTER .flt
   #</Directory>
   
   # This will enable use of "Python Server Pages", HTML pages with
   # inline Python code which is interpreted by the server:
   #
   #<Directory /var/www/html>
   #    AddHandler mod_python .psp
   #    PythonHandler mod_python.psp
   #</Directory>
   
   

7. Restart the httpd service
   
8. Set up the profile scripts to use the new python and trac
   

   
   $ sudo vim /etc/profile.d/trac-12.sh
   
   

   
   #!/bin/sh
   alias python="/opt/python/bin/python2.6"
   export PATH=/opt/python/bin:$PATH
   
   

   
   #!csh
   $ sudo vim /etc/profile.d/trac-12.csh
   
   

   
   #!/bin/tcsh
   alias python '/opt/python/bin/python2.6'
   setenv PATH "/opt/python/bin:${PATH}"
   
   

Subversion

This section documents the installation of Subversion 1.6.x. CentOS 5 comes with 1.4.x.

1. Create the install location
   

   
   $ sudo mkdir /opt/subversion
   $ sudo chown -R flakrat:flakrat /opt/subversion
   
   

2. Download Subversion
   

   
   $ mkdir -p ~/software/subversion
   $ cd ~/software/subversion
   $ wget http://subversion.tigris.org/downloads/subversion-1.6.11.tar.bz2
   $ wget http://subversion.tigris.org/downloads/subversion-deps-1.6.11.tar.bz2
   $ tar -jxf subversion-1.6.11.tar.bz2
   $ tar -jxf subversion-deps-1.6.11.tar.bz2
   
   

3. Compile Subversion (during 'make check' XFAILs are expected failures that are sort of like a TODO for the SVN developers), the 'make swig-py' will build the SVN Python libraries
   

   
   $ . /etc/profile.d/trac-12.sh
   $ cd subversion-1.6.11
   $ make clean
   $ sh ./autogen.sh
   
   $ ./configure \
     --prefix=/opt/subversion \
     --with-apr=/usr/bin/apr-1-config \
     --with-apr-util=/usr/bin/apu-1-config \
     --with-apxs=/usr/sbin/apxs
   
   $ make
   $ make check
   $ sudo make install
   
   $ make swig-py
   $ sudo make install-swig-py 
   
   

4. Copy the Python libraries to the Python directory
   

   
   $ cp -a /opt/subversion/lib/svn-python/{svn,libsvn} \
     /opt/python/lib/python2.6/site-packages/
   
   

5. Create the profile scripts
   * Bash
   

   
   $ sudo vim /etc/profile.d/svn.sh
   
   

   
   #!/bin/sh
   export PATH=/opt/subversion/bin:$PATH
   export LD_LIBRARY_PATH=/opt/subversion/lib:$LD_LIBRARY_PATH
   
   

   * CSH
   

   
   $ sudo vim /etc/profile.d/svn.csh
   
   

   
   #!/bin/tcsh
   setenv PATH "/opt/subversion/bin:${PATH}"
   setenv LD_LIBRARY_PATH "/opt/subversion/lib:${LD_LIBRARY_PATH}"
   
   

6. Edit the yum.conf file to exclude subverion and mod_dav_svn from being installed
   

   
   $ sudo vim /etc/yum.conf
   
   

   
   [main]
   cachedir=/var/cache/yum
   keepcache=0
   debuglevel=2
   logfile=/var/log/yum.log
   distroverpkg=redhat-release
   tolerant=1
   exactarch=1
   obsoletes=1
   gpgcheck=1
   plugins=1
   
   exclude=subversion,mod_dav_svn
   
   # Note: yum-RHN-plugin doesn't honor this.
   #metadata_expire=1h
   
   # Default.
   # installonly_limit = 3
   
   # PUT YOUR REPOS HERE OR IN separate files named file.repo
   # in /etc/yum.repos.d
   

7. The Subversion repositories on the old svn server used BDB format. Newer version of Subversion use FSFS as the standard. In order to facilitate migrating the repos to the new server I had to first dump the repos (these commands were run on my old server where repos were stored in ~svn/repositories):
   

   
   $ cd ~svn/repositories
   $ for n in $(ls); do \
    if [ -d $n ]; then echo "Dumping: $n"; \
      svnadmin dump ${n} | gzip -c > /backups/dumps/${n}.dmp.gz ; \
   fi; done
   

8. Create md5sum checksums of each dump file (also run on the old server)
   

   
   $ cd /backups/dumps
   $ find $@ ! -type d -print0 | xargs -0 md5sum > /backups/MD5SUMS.svn.dumps 
   

9. Copy the dumps and check sum file from the old server and verify the dump files (commands run on the new server)
   

   
   $ mkdir ~svn/dumps
   $ rsync -a oldserver:/backups/dumps/ ~svn/dumps/
   $ rsync -a oldserver:/backups/MD5SUMS.svn.dumps ~svn/
   $ cd ~svn/dumps
   $ md5sum -c ~svn/MD5SUMS.svn.dumps | grep -v OK$
   

10. If everything checks out, the md5sum command shouldn't produce any output since we are greping out lines ending in OK
    
11. Load the Subversion repositories from their gzip compressed dump files
    

    
    $ cd ~svn/dumps
    $ sudo mkdir -p /var/lib/svn/repos
    $ sudo chown -R svn:svnadmin /var/lib/svn
    $ sudo chmod -R g+rwxs /var/lib/svn
    
    $ for n in $(ls | awk -F. '{print $1}'); do \
      svnadmin create /var/lib/svn/repos/$n ; \
      gunzip -c ${n}.dmp.gz | svnadmin load /var/lib/svn/repos/$n ; \
    done
    

12. Next verify the repositories
    

    
    $ cd /var/lib/svn/repos
    $ for n in $(ls); do svnadmin verify $n ; done
    
    

Trac

1. Source the trac environment script
   

   
   $ . /etc/profile.d/trac-12.sh
   
   

2. Install Genshi
   

   
   $ easy_install Genshi
   
   Searching for Genshi
   Reading http://pypi.python.org/simple/Genshi/
   Reading http://genshi.edgewall.org/
   Reading http://genshi.edgewall.org/wiki/Download
   Best match: Genshi 0.6
   Downloading http://ftp.edgewall.com/pub/genshi/Genshi-0.6-py2.6.egg
   Processing Genshi-0.6-py2.6.egg
   Moving Genshi-0.6-py2.6.egg to /opt/python/lib/python2.6/site-packages
   Adding Genshi 0.6 to easy-install.pth file
   
   Installed /opt/python/lib/python2.6/site-packages/Genshi-0.6-py2.6.egg
   Processing dependencies for Genshi
   Finished processing dependencies for Genshi
   
   

3. Install Trac 0.12 using easy_install
   

   
   $ easy_install Trac==0.12
   
   Searching for Trac==0.12
   Reading http://pypi.python.org/simple/Trac/
   Reading http://trac.edgewall.org/
   Reading http://trac.edgewall.org/wiki/TracDownload
   Reading http://trac.edgewall.com/
   Reading http://projects.edgewall.com/trac
   Reading http://projects.edgewall.com/trac/wiki/TracDownload
   Best match: Trac 0.12
   Downloading ftp://ftp.edgewall.com/pub/trac/Trac-0.12.zip
   Processing Trac-0.12.zip
   Running Trac-0.12/setup.py -q bdist_egg --dist-dir /tmp/easy_install-m7d5G7/Trac-0.12/egg-dist-tmp-gqKlcr
   Adding Trac 0.12 to easy-install.pth file
   Installing trac-admin script to /opt/python/bin
   Installing tracd script to /opt/python/bin
   
   Installed /opt/python/lib/python2.6/site-packages/Trac-0.12-py2.6.egg
   Processing dependencies for Trac==0.12
   Finished processing dependencies for Trac==0.12
   
   

4. Install the Git plugin (for future support of Git repos on this server)
   

   
   $ easy_install http://trac-hacks.org/svn/gitplugin/0.12
   
   Downloading http://trac-hacks.org/svn/gitplugin/0.12
   Doing subversion checkout from http://trac-hacks.org/svn/gitplugin/0.12 to /tmp/easy_install-9ULtKC/0.12
   Processing 0.12
   Running setup.py -q bdist_egg --dist-dir /tmp/easy_install-9ULtKC/0.12/egg-dist-tmp-iTzr9l
   warning: install_data: setup script did not provide a directory for 'COPYING' -- installing right in 'build/bdist.linux-i686/egg'
   warning: install_data: setup script did not provide a directory for 'README' -- installing right in 'build/bdist.linux-i686/egg'
   zip_safe flag not set; analyzing archive contents...
   Adding TracGit 0.12.0.2dev-r7757 to easy-install.pth file
   
   Installed /opt/python/lib/python2.6/site-packages/TracGit-0.12.0.2dev_r7757-py2.6.egg
   Processing dependencies for TracGit==0.12.0.2dev-r7757
   Finished processing dependencies for TracGit==0.12.0.2dev-r7757
   
   

5. Install TOCMacro http://trac-hacks.org/wiki/TocMacro (the version for 0.11 appears to work, we'll use it until 0.12 toc releases)
   

   
   $ easy_install http://trac-hacks.org/svn/tocmacro/0.11
   
   

6. Install pygments for syntax highlighting
   

   
   $ easy_install pygments
   Searching for pygments
   Reading http://pypi.python.org/simple/pygments/
   Reading http://pygments.org/
   Reading http://pygments.pocoo.org/
   Best match: Pygments 1.3.1
   Downloading http://pypi.python.org/packages/2.6/P/Pygments/Pygments-1.3.1-py2.6.egg#md5=05270ee059c038407364b28993980960
   Processing Pygments-1.3.1-py2.6.egg
   creating /opt/python/lib/python2.6/site-packages/Pygments-1.3.1-py2.6.egg
   Extracting Pygments-1.3.1-py2.6.egg to /opt/python/lib/python2.6/site-packages
   Adding Pygments 1.3.1 to easy-install.pth file
   Installing pygmentize script to /opt/python/bin
   
   Installed /opt/python/lib/python2.6/site-packages/Pygments-1.3.1-py2.6.egg
   Processing dependencies for pygments
   Finished processing dependencies for pygments
   
   

7. Change ownership of the python install
   

   
   $ sudo chown -R root:root /opt/python
   
   

Trac Pages

This section will document how to configure trac and convert each to use the updated format for version 0.12

1. Copy the old trac instances to /var/www/trac (during the install I staged all of the files in /backup/oldsrv)
   

   
   $ sudo mkdir /var/www/trac
   $ sudo rsync -a /backup/oldsrv/var/www/trac/ /var/www/trac/
   $ sudo chown -R apache:apache /var/www/trac
   
   

2. Upgrade the trac instances to the current format
   

   
   $ for n in $(ls); do \
     sudo trac-admin /var/www/trac/$n upgrade; \
     sudo trac-admin /var/www/trac/$n wiki upgrade; \
     sudo trac-admin /var/www/trac/$n repository '*'; \
    done
   
   

3. Create the trac.conf httpd file
   

   
   $ sudo vim /etc/httpd/conf.d/trac.conf
   
   

   
   <Location /trac>
     SetHandler mod_python
     PythonHandler trac.web.modpython_frontend
     PythonOption TracEnvParentDir /var/www/trac
     PythonOption TracUriRoot /trac
   </Location>
   
   <LocationMatch /trac/[^/]+/login>
     AuthType Basic
     AuthName "trac"
     AuthUserFile /var/lib/svn/repos/repousers
     Require valid-user
   </LocationMatch>
   
   

iPhone IOS 4 Initial Thoughts

Here are a few initial impressions of iPhone IOS 4 update on a 3G:

• Track skip forward and back via Bluetooth head set still does not work, at least on my Motorola stereo head set
• There still isn't an option to 'snooze' calendar reminders
• Spell check is finally included and works as expected (red underline)
• Unified email inbox is nice
• Email threading is very handy
• The IOS 4 ran slow as molasses on my 3G

As a test I restored the phone to factory defaults. The iPhone was very responsive in the default config. I restored from the most recent backup, but chose not to sync any applications and again the iPhone was responsive. I then synced the handful of apps that I use regularly and performance is still good.

Previously, I had six pages of apps, now only two. This isn't conclusive by any stretch, but for now the 3G is much more tolerable than it was.

I still need to re-install all of the apps to test the responsiveness, but if your 3G is behaving similarly, perhaps simply reducing the number of apps might help.

How To: Enable Broadcom BCM4312 wifi in RHEL 6

Howdy,

I figured I'd jot down the steps to enable the Broadcom BCM4312 802.11b/g wifi card in Red Hat Enterprise Linux 6 beta x86_64 (RHEL6).

Broadcom wireless chipsets are very common in laptops. The laptop in this example is a Dell Latitude D630. Broadcom wifi has been a pain to enable in the past, especially in RHEL4.

As you can see in this posting on kernel.org things get a lot easier in kernels later than 2.6.24

First install the b43 packages (Edit: removed b43-tools as it is not a valid package)

$ sudo yum install b43-fwcutter b43-openfwwf

Next verify that your device is supported

$ sudo /sbin/lspci -vnn | grep 14e4

09:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM5755M Gigabit Ethernet PCI Express [14e4:1673] (rev 02)
0c:00.0 Network controller [0280]: Broadcom Corporation BCM4312 802.11b/g [14e4:4315] (rev 01)

Using the full number, in this case 14e4:4315, look it up in this table to ensure that it is supported and which kernel is required.

In this case it should be supported since RHEL 6 beta uses kernel 2.6.32-19 and the minimum requirements listed in the table are kernel 2.6.32. Barely made it :-)

14e4:4315 supported 2.6.32 and later  BCM4312  b/g  LP  b43

Download the Windows drivers

$ mkdir ~/drivers
$ cd ~/drivers
$ wget http://downloads.openwrt.org/sources/broadcom-wl-4.150.10.5.tar.bz2

Extract and copy the firmware to /lib/firmware

$ tar -jxf broadcom-wl-4.150.10.5.tar.bz2
$ cd broadcom-wl-4.150.10.5/driver
$ sudo /usr/bin/b43-fwcutter -w /lib/firmware/ wl_apsta_mimo.o 

This file is recognised as:
  ID         :  FW13
  filename   :  wl_apsta_mimo.o
  version    :  410.2160
  MD5        :  cb8d70972b885b1f8883b943c0261a3c
Extracting b43/pcm5.fw
Extracting b43/ucode15.fw
Extracting b43/ucode14.fw
Extracting b43/ucode13.fw
Extracting b43/ucode11.fw
Extracting b43/ucode9.fw
Extracting b43/ucode5.fw
Extracting b43/lp0bsinitvals15.fw
Extracting b43/lp0initvals15.fw
Extracting b43/lp0bsinitvals14.fw
Extracting b43/lp0initvals14.fw
Extracting b43/a0g1bsinitvals13.fw
Extracting b43/a0g1initvals13.fw
Extracting b43/b0g0bsinitvals13.fw
Extracting b43/b0g0initvals13.fw
Extracting b43/lp0bsinitvals13.fw
Extracting b43/lp0initvals13.fw
Extracting b43/n0absinitvals11.fw
Extracting b43/n0bsinitvals11.fw
Extracting b43/n0initvals11.fw
Extracting b43/a0g1bsinitvals9.fw
Extracting b43/a0g0bsinitvals9.fw
Extracting b43/a0g1initvals9.fw
Extracting b43/a0g0initvals9.fw
Extracting b43/b0g0bsinitvals9.fw
Extracting b43/b0g0initvals9.fw
Extracting b43/a0g1bsinitvals5.fw
Extracting b43/a0g0bsinitvals5.fw
Extracting b43/a0g1initvals5.fw
Extracting b43/a0g0initvals5.fw
Extracting b43/b0g0bsinitvals5.fw
Extracting b43/b0g0initvals5.fw

Next toggle the wifi radio switch, once it's switched back to the on position, the wifi light should begin flashing.

The lsmod command should now list the b43 module

$ /sbin/lsmod | grep -i b43
b43                   204044  0 
mac80211              231588  1 b43
cfg80211              142707  2 b43,mac80211
ssb                    56886  1 b43
mmc_core               72904  2 b43,ssb

At this point you can either add the wlan0 device manually, or you can reboot and allow the system to detect and configure it.

Following the reboot (or adding the device manually) you should now see wlan0 in the ifconfig output, and see available networks via Network Manager.

Spacewalk 1.0 has Released

The Spacewalk developers have released 1.0 of Spacewalk, the "open source Linux and Solaris systems management solution".

The release notes are available here. Recent previous version were 0.8, 0.7 and 0.6.