FlakRat's Rat House

Friday, April 19, 2013

How To Compile Gromacs 4.6.1 with OpenMPI 1.6.4 on CentOS

I took the following notes while installing Gromacs 4.6.1 recently. Hopefully others will find them useful. The software was compiled and installed on a CentOS 5.9 x86_64 containing QDR Infiniband. The system uses SGE (Grid Engine) as it's scheduler. Intel compilers 13.1.1 were used to compile both OpenMPI and Gromacs.

Intel Compilers

I used version 13.1.1 of the Intel compilers. I'm including the module file here for reference

#%Module

set ver 13.1.1
set arch intel64

set root /share/apps/intel/parallel_studio_xe_2013_update3/composer_xe_2013.3.163
set ccroot /opt/intel/Compiler/$ver/$ccver
set fcroot /opt/intel/Compiler/$ver/$fcver
set mklroot $root/mkl

set msg "This module adds Intel compilers v$ver to various paths"

proc ModulesHelp { } {
   puts stderr $msg
}

module-whatis   $msg

setenv MKLROOT $mklroot
prepend-path MANPATH $root/man/en_US
prepend-path INTEL_LICENSE_FILE $root/licenses

prepend-path NLSPATH $root/compiler/lib/$arch/locale/en_US:$root/ipp/lib/$arch/locale/en_US:$root/mkl/lib/$arch/locale/en_US:$root/debugger/lib/$arch/locale/en_US

prepend-path LIBRARY_PATH $root/compiler/lib/$arch:$root/ipp/lib/$arch:$root/mkl/lib/$arch:$root/tbb/lib/$arch/gcc4.1
prepend-path MIC_LD_LIBRARY_PATH $root/compiler/lib/mic:$root/mkl/lib/mic:$root/tbb/lib/mic
prepend-path LD_LIBRARY_PATH $root/compiler/lib/$arch:$root/mpirt/lib/$arch:$root/ipp/lib/$arch:$root/mkl/lib/$arch:$root/tbb/lib/$arch/gcc4.1
prepend-path PATH $root/bin/$arch:$root/mpirt/bin/$arch:$root/bin/${arch}_mic:$root/debugger/gui/$arch

prepend-path CPATH $root/mkl/include:$root/tbb/include
prepend-path INCLUDE $root/mkl/include

prepend-path IPPROOT $root/ipp
prepend-path TBBROOT $root/tbb
prepend-path IDB_HOME $root/bin/$arch

Build OpenMPI 1.6.4

First, build the latest OpenMPI with the Intel compilers. This version of OpenMPI will reside in the Gromacs 4.6.1 install path.

Create the install base directory

$ cd /share/apps/gromacs/4.6.1/build
$ wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.4.tar.bz2

Create the build script for Intel compilers: /share/apps/gromacs/4.6.1/build/build-openmpi-intel.sh

#!/bin/sh
. /etc/profile.d/modules.sh
ver=1.6
rel=1.6.4
grover=4.6.1
src="openmpi-${rel}.tar.bz2"
url="http://www.open-mpi.org/software/ompi/v${ver}/downloads/${src}"
compiler="intel"
basedir="/share/apps/gromacs/${grover}/openmpi"
prefix="${basedir}/${rel}-${compiler}"

if [ -d $prefix ]; then
  echo "Installation directory already exists, exiting to prevent overwrite: $prefix";
  exit 1
else
  echo "Verified installation directory does not exist: $prefix"
fi

if [ ! -f $src ]; then
  echo "Downloading source code from: $url";
  wget $url;
else
  echo "Using existing source code: $src";
fi

if [ -d "openmpi-${rel}" ]; then
  echo "Removing existing build directory ./openmpi-${rel}"
  rm -rf openmpi-${rel};
fi

echo "Extracting $src"
tar -jxf $src
cd openmpi-${rel}

echo "Building OpenMPI $rel for Intel compilers"
echo "Running configure"
module purge
module load intel/intel-compilers-13.1.1
CC=icc CXX=icpc F77=ifort FC=ifort ./configure --with-sge --with-openib --prefix=$prefix --enable-static --enable-shared

make all install
module purge

cd ..

Run the compile and install

$ ./build-openmpi-intel.sh | tee openmpi-1.6.4-intel.log

Create the module file (based off of [http://www.levlafayette.com/node/145 this example]) in the Gromacs base directory

$ mkdir -p /share/apps/gromacs/4.6.1/modulefiles/gromacs-openmpi-intel
$ vim /share/apps/gromacs/4.6.1/modulefiles/gromacs-openmpi-intel/1.6.4

#%Module1.0#####################################################################
##
## $name modulefile
##
# /share/apps/gromacs/4.6.1/openmpi/1.6.4-intel/bin
# /share/apps/gromacs/4.6.1/openmpi/gromacs-openmpi-intel/1.6.4-intel/bin
set ver [lrange [split [ module-info name ] / ] 1 1 ]
set name [lrange [split [ module-info name ] / ] 0 0 ]
set loading [module-info mode load]
set subname [lrange [split $name - ] 0 0 ]
#set compiler [lrange [split $name - ] 1 1 ]
set compiler "intel"
set compilerver "13.1.1"

proc ModulesHelp { } {
  puts stderr "\tThis module sets the envinronment for $name v$ver compiled\n\twith Intel v13.1.1 compilers"
}

module-whatis   "Set environment variables to use $name version $ver"

if { $loading && ![ is-loaded $compiler/$compilerver ] } {
  module load $compiler/$compilerver
}

set basedir /share/apps/gromacs/4.6.1/openmpi/${ver}-$compiler

prepend-path --delim " " CPPFLAGS -I$basedir/include
prepend-path --delim " " LDFLAGS -L$basedir/lib

prepend-path LD_LIBRARY_PATH $basedir/lib
prepend-path MANPATH $basedir/share/man
prepend-path PATH $basedir/bin

setenv  MPI_DIR         $basedir/
setenv  MPI_HOME        $basedir/
setenv  OPENMPI_ROOT    $basedir/

Run a test

Create the input C file /share/apps/openmpi/build/mpi-helloworld.c

#include <stdio>
#include <string>
#include <stddef>
#include <stdlib>
#include "mpi.h"

main(int argc, char **argv ) {
    int rank, size;
    int buflen = 512;
    char name[buflen];
    
    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD,&size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    gethostname(name, buflen);
    
    printf( "P rank %d of %d, host %s\n", rank, size, name);

    MPI_Finalize();
}

Compile mpi-helloworld.c

$ module load /share/apps/gromacs/4.6.1/modulefiles/gromacs-openmpi-intel/1.6.4
$ mpicc -o mpi-helloworld-intel mpi-helloworld.c

run it as a job

$ module purge
$ module load /share/apps/gromacs/4.6.1/modulefiles/gromacs-openmpi-intel/1.6.4
$ qsub -V -b y \
   -l h_rt=00:15:00 \
   -l vf=256M \
   -j y \
   -pe openmpi\* 8 \
   mpirun -n 8 /share/apps/openmpi/build/mpi-helloworld-intel

Build Gromacs 4.6.1

Official Gromacs 4.6 build instructions They have changed dramatically from versions 4.5 and prior. One requirement listed is that we need CMake version 2.8 or later. EPEL provides CMake 2.6.4, so we'll need to build CMake from scratch.

Download the latest CMake from here: http://www.cmake.org/cmake/resources/software.html

sudo mkdir /share/apps/cmake
sudo chown mhanby:atlab /share/apps/cmake
cd /share/apps/cmake
wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2-Linux-i386.tar.gz
tar -zxf cmake-2.8.10.2-Linux-i386.tar.gz

Create the build directory and download the source

mkdir -p /share/apps/gromacs/4.6.1/build
cd /share/apps/gromacs/4.6.1/build
wget ftp://ftp.gromacs.org/pub/gromacs/gromacs-4.6.1.tar.gz
tar -zxf gromacs-4.6.1.tar.gz 
mkdir gromacs-4.6.1/build
cd gromacs-4.6.1/build

Create the module files

$ mkdir /share/apps/modulefiles/gromacs-intel
$ vim /share/apps/modulefiles/gromacs-intel/4.6.1

#%Module

set ver 4.6.1
set app "Gromacs"
set url "http://gromacs.org/"
set approot /share/apps/gromacs/$ver
set msg "This module adds $app v$ver to various paths\nWebsite: $url\n\n"

proc ModulesHelp { } {
   puts stderr $msg
}

module-whatis  $msg

module use --append $approot/modulefiles
module load intel/13.1.1
#module load $approot/modulefiles/gromacs-openmpi-intel/1.6.4
module load gromacs-openmpi-intel/1.6.4

setenv GROMACSHOME $approot
#setenv MKL_HOME $::env(MKLROOT)
prepend-path PATH $approot/bin
prepend-path LD_LIBRARY_PATH $approot/lib
prepend-path MANPATH $approot/man
prepend-path MODULEPATH $approot/modulefiles
prepend-path PKG_CONFIG_PATH $approot/lib/pkgconfig

The next steps will build single and double precision in serial and parallel

Serial single precision

Run cmake to configure the build

module purge
module load gromacs-intel/4.6.1
export PATH=/share/apps/cmake/cmake-2.8.10.2-Linux-i386/bin:$PATH

cd /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
rm -rf *

cmake .. \
   -DCMAKE_C_COMPILER=icc \
   -DCMAKE_CXX_COMPILER=icpc \
   -DGMX_BUILD_OWN_FFTW=ON \
   -DCMAKE_INSTALL_PREFIX=/share/apps/gromacs/4.6.1 \
   -DREGRESSIONTEST_DOWNLOAD=ON

View the configuration

$ ccmake ..

 BUILD_SHARED_LIBS                ON
 CMAKE_BUILD_TYPE                 Release
 CMAKE_INSTALL_PREFIX             /share/apps/gromacs/4.6.1
 CMAKE_PREFIX_PATH
 CUDA_HOST_COMPILER               /share/apps/intel/parallel_studio_xe_2013_update3/composer_xe_2013.3.163/bin/intel64/icc
 GMX_CPU_ACCELERATION             SSE4.1
 GMX_DEFAULT_SUFFIX               ON
 GMX_DOUBLE                       OFF
 GMX_FFT_LIBRARY                  fftw3
 GMX_GPU                          OFF
 GMX_GSL                          OFF
 GMX_MPI                          OFF
 GMX_OPENMP                       ON
 GMX_QMMM_PROGRAM                 none
 GMX_THREAD_MPI                   ON
 GMX_X11                          OFF
 REGRESSIONTEST_DOWNLOAD          OFF

make
```
$ make
```

Run the tests

$ make check
[ 70%] Built target gmx
[ 71%] Built target gmxfftw
[ 80%] Built target md
[ 92%] Built target gmxana
[ 92%] Built target editconf
[ 96%] Built target gmxpreprocess
[ 97%] Built target grompp
[ 97%] Built target pdb2gmx
[ 97%] Built target gmxcheck
[100%] Built target mdrun
[100%] Built target gmxtests
Test project /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
    Start 1: regressiontests/simple
1/5 Test #1: regressiontests/simple ...........   Passed   72.17 sec
    Start 2: regressiontests/complex
2/5 Test #2: regressiontests/complex ..........   Passed  868.76 sec
    Start 3: regressiontests/kernel
3/5 Test #3: regressiontests/kernel ...........   Passed  583.48 sec
    Start 4: regressiontests/freeenergy
4/5 Test #4: regressiontests/freeenergy .......   Passed  695.84 sec
    Start 5: regressiontests/pdb2gmx
5/5 Test #5: regressiontests/pdb2gmx ..........   Passed  379.29 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) = 2599.69 sec
[100%] Built target check

Install

$ make install

...
-- Installing: /share/apps/gromacs/4.6.1/lib/pkgconfig/libgmxana.pc
-- Installing: /share/apps/gromacs/4.6.1/bin/GMXRC
-- Installing: /share/apps/gromacs/4.6.1/bin/GMXRC.bash
-- Installing: /share/apps/gromacs/4.6.1/bin/GMXRC.zsh
-- Installing: /share/apps/gromacs/4.6.1/bin/GMXRC.csh
-- Installing: /share/apps/gromacs/4.6.1/bin/completion.csh
-- Installing: /share/apps/gromacs/4.6.1/bin/completion.bash
-- Installing: /share/apps/gromacs/4.6.1/bin/completion.zsh
-- Installing: /share/apps/gromacs/4.6.1/bin/demux.pl
-- Installing: /share/apps/gromacs/4.6.1/bin/xplor2gmx.pl

Serial double precision (will append _d to applicable installed files, example, mdrun_d)

Run cmake to configure the build

module purge
module load gromacs-intel/4.6.1
export PATH=/share/apps/cmake/cmake-2.8.10.2-Linux-i386/bin:$PATH

cd /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
rm -rf *

cmake .. \
   -DCMAKE_C_COMPILER=icc \
   -DCMAKE_CXX_COMPILER=icpc \
   -DGMX_BUILD_OWN_FFTW=ON \
   -DGMX_DOUBLE=ON \
   -DCMAKE_INSTALL_PREFIX=/share/apps/gromacs/4.6.1 \
   -DREGRESSIONTEST_DOWNLOAD=ON

View the configuration

$ ccmake ..

 BUILD_SHARED_LIBS                ON
 CMAKE_BUILD_TYPE                 Release
 CMAKE_INSTALL_PREFIX             /share/apps/gromacs/4.6.1
 CMAKE_PREFIX_PATH
 GMX_CPU_ACCELERATION             SSE4.1
 GMX_DEFAULT_SUFFIX               ON
 GMX_DOUBLE                       ON
 GMX_FFT_LIBRARY                  fftw3
 GMX_GPU                          OFF
 GMX_GSL                          OFF
 GMX_MPI                          OFF
 GMX_OPENMP                       ON
 GMX_QMMM_PROGRAM                 none
 GMX_THREAD_MPI                   ON
 GMX_X11                          OFF
 REGRESSIONTEST_DOWNLOAD          OFF

make
```
$ make
```

Run the tests

$ make check

Test project /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
    Start 1: regressiontests/simple
1/5 Test #1: regressiontests/simple ...........   Passed    3.33 sec
    Start 2: regressiontests/complex
2/5 Test #2: regressiontests/complex ..........   Passed    6.80 sec
    Start 3: regressiontests/kernel
3/5 Test #3: regressiontests/kernel ...........   Passed   52.89 sec
    Start 4: regressiontests/freeenergy
4/5 Test #4: regressiontests/freeenergy .......   Passed    9.19 sec
    Start 5: regressiontests/pdb2gmx
5/5 Test #5: regressiontests/pdb2gmx ..........   Passed   22.82 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) =  95.08 sec
[100%] Built target check

Install
```
$ make install
```

Parallel single precision (will append _mpi to applicable installed files, example, mdrun_mpi)

Run cmake to configure the build

module purge
module load gromacs-intel/4.6.1
export PATH=/share/apps/cmake/cmake-2.8.10.2-Linux-i386/bin:$PATH

cd /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
rm -rf *

cmake .. \
   -DCMAKE_C_COMPILER=icc \
   -DCMAKE_CXX_COMPILER=icpc \
   -DGMX_BUILD_OWN_FFTW=ON \
   -DGMX_MPI=ON \
   -DCMAKE_INSTALL_PREFIX=/share/apps/gromacs/4.6.1 \
   -DREGRESSIONTEST_DOWNLOAD=ON

View the configuration

$ ccmake ..

 BUILD_SHARED_LIBS                ON
 CMAKE_BUILD_TYPE                 Release
 CMAKE_INSTALL_PREFIX             /share/apps/gromacs/4.6.1
 CMAKE_PREFIX_PATH
 CUDA_HOST_COMPILER               /share/apps/intel/parallel_studio_xe_2013_update3/composer_xe_2013.3.163/bin/intel64/icc
 GMX_CPU_ACCELERATION             SSE4.1
 GMX_DEFAULT_SUFFIX               ON
 GMX_DOUBLE                       OFF
 GMX_FFT_LIBRARY                  fftw3
 GMX_GPU                          OFF
 GMX_GSL                          OFF
 GMX_MPI                          ON
 GMX_OPENMP                       ON
 GMX_QMMM_PROGRAM                 none
 GMX_THREAD_MPI                   OFF
 GMX_X11                          OFF
 MPI_EXTRA_LIBRARY                /usr/lib64/librdmacm.so;/usr/lib64/libibverbs.so;/usr/lib64/librt.so;/usr/lib64/libnsl.so;/usr/lib64/libutil.so;/usr/lib64/libdl.so;/usr/lib64/libm.so;/usr/lib64/librt.so;
 MPI_LIBRARY                      /share/apps/gromacs/4.6.1/openmpi/1.6.4-intel/lib/libmpi.so
 REGRESSIONTEST_DOWNLOAD          OFF

make
```
$ make
```

Run the tests

$ make check

Test project /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
    Start 1: regressiontests/simple
1/5 Test #1: regressiontests/simple ...........   Passed   22.68 sec
    Start 2: regressiontests/complex
2/5 Test #2: regressiontests/complex ..........   Passed   29.49 sec
    Start 3: regressiontests/kernel
3/5 Test #3: regressiontests/kernel ...........   Passed  222.08 sec
    Start 4: regressiontests/freeenergy
4/5 Test #4: regressiontests/freeenergy .......   Passed   19.17 sec
    Start 5: regressiontests/pdb2gmx
5/5 Test #5: regressiontests/pdb2gmx ..........   Passed   75.39 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) = 368.83 sec
[100%] Built target check

Install
```
$ make install
```

Parallel double precision (will append _mpi_d to applicable installed files, example, mdrun_mpi_d)

Run cmake to configure the build

module purge
module load gromacs-intel/4.6.1
export PATH=/share/apps/cmake/cmake-2.8.10.2-Linux-i386/bin:$PATH

cd /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
rm -rf *

cmake .. \
   -DCMAKE_C_COMPILER=icc \
   -DCMAKE_CXX_COMPILER=icpc \
   -DGMX_BUILD_OWN_FFTW=ON \
   -DGMX_MPI=ON \
   -DGMX_DOUBLE=ON \
   -DCMAKE_INSTALL_PREFIX=/share/apps/gromacs/4.6.1 \
   -DREGRESSIONTEST_DOWNLOAD=ON

View the configuration

$ ccmake ..

 BUILD_SHARED_LIBS                ON 
 CMAKE_BUILD_TYPE                 Release
 CMAKE_INSTALL_PREFIX             /share/apps/gromacs/4.6.1
 CMAKE_PREFIX_PATH
 GMX_CPU_ACCELERATION             SSE4.1
 GMX_DEFAULT_SUFFIX               ON
 GMX_DOUBLE                       ON
 GMX_FFT_LIBRARY                  fftw3
 GMX_GPU                          OFF
 GMX_GSL                          OFF
 GMX_MPI                          ON
 GMX_OPENMP                       ON
 GMX_QMMM_PROGRAM                 none
 GMX_THREAD_MPI                   OFF
 GMX_X11                          OFF
 MPI_EXTRA_LIBRARY                /usr/lib64/librdmacm.so;/usr/lib64/libibverbs.so;/usr/lib64/librt.so;/usr/lib64/libnsl.so;/usr/lib64/libutil.so;/usr/lib64/libdl.so;/usr/lib64/libm.so;/usr/lib64/librt.so;/usr/lib64/libnsl.so;/usr/lib64/libutil.so                       
 MPI_LIBRARY                      /share/apps/gromacs/4.6.1/openmpi/1.6.4-intel/lib/libmpi.so
 REGRESSIONTEST_DOWNLOAD          OFF

make
```
$ make
```

Run the tests

$ make check

Test project /share/apps/gromacs/4.6.1/build/gromacs-4.6.1/build
    Start 1: regressiontests/simple
1/5 Test #1: regressiontests/simple ...........   Passed   22.65 sec
    Start 2: regressiontests/complex
2/5 Test #2: regressiontests/complex ..........   Passed   32.25 sec
    Start 3: regressiontests/kernel
3/5 Test #3: regressiontests/kernel ...........   Passed  232.23 sec
    Start 4: regressiontests/freeenergy
4/5 Test #4: regressiontests/freeenergy .......   Passed   21.13 sec
    Start 5: regressiontests/pdb2gmx
5/5 Test #5: regressiontests/pdb2gmx ..........   Passed   82.39 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) = 390.67 sec
[100%] Built target check

Install
```
$ make install
```

Wednesday, February 27, 2013

Ubuntu 13.04 on Google Nexus 7

Canonical has recently released a developer preview of Ubuntu 13.04 for Google Nexus devices.

I figured, what the heck, may as well try it out on my Google Nexus 7. Following these instructions, the installation went off without a hitch.

I used a laptop running Ubuntu 12.10 to perform the installation.

Following the installation, I ran into the issue during First Boot where the on screen keyboard didn't work. I don't have an OTG adapter cable, but do have a Bluetooth Logitech Android tablet keyboard. As stated in the instructions, the Bluetooth keyboard doesn't work during first boot.

That is, initially :-)

I got around this by clicking the Gear icon in the upper right corner and clicking System Settings -> Bluetooth. I clicked the add button and put the keyboard into connect mode. The keyboard successfully synced with the tablet.

Great, however now I couldn't find a way to close the System Settings window. I could move it around, but couldn't get the focus away and back to the First Boot screen.

No problem, click the Gear icon again and select restart. Upon reboot, the Bluetooth keyboard works fine on the First Boot windows.

Wednesday, January 23, 2013

How To: Recursively Create Numeric Directories

I was asked by a colleague how to do the following:


I have a root folder that I need to create a directory based on job numbers.

From the root folder, I'd like to create the following:

1000
1000\1100
1000\1100\1101
1000\1100\1102

all the way up to 

9000\9900\9999

To explain, I'll use the 5000 folder:

I would need:
5000
5000\5100
5000\5100\5101 through 5000\5100\5199
5000\5200
5000\5200\5201 through 5200\5200\5299
and so on.

What would be the easiest way?

The solution isn't a big deal for anyone familiar with scripting, but I figured I'd share here in case other none coders need help with a similar problem.

I wasn't sure whether they were on a Windows computer (figured most likely based on the slashes) or Linux / Mac, so I provided both a Bash and Windows command shell solutions.

There are any number of ways to approach this, I chose nested for loops.

First the Bash solution:

for x in {1..9}; do
  mkdir ${x}000
  cd ${x}000
  for y in {1..9}; do
    mkdir ${x}${y}00
    cd ${x}${y}00
    for ((z=1; z<=99; z+=1)); do
      mkdir `printf "%s%s%02d" $x $y $z`
    done
    cd ..
  done
  cd ..
done

Now the Windows solution:

setlocal EnableDelayedExpansion
for /l %x in (1, 1, 9) do (
  mkdir %x000
  cd %x000
  for /l %y in (1, 1, 9) do (
    mkdir %x%y00
    cd %x%y00
    for /l %z in (1, 1, 99) do (
      if %z LSS 10 (
        mkdir %x%y0%z
      ) else (
        mkdir %x%y%z
      )
    )
    cd ..
  )
  cd ..  
)

Wednesday, June 20, 2012

Build Firewall Builder 'FWbuilder' 5.1.0 for Fedora 17

Howdy, FirewallBuilder released version 5.1.0 without providing an officially built rpm for Fedora 17. This post will build an rpm for Fedora 17 using the src rpm for Fedora 16 If you need help configuring the rpmbuild environment, start here. First, download the fwbuilder-5.1.0.3599-1.fc16.src.rpm file to your local directory. Next, extract the src.rpm

$ cd ~/tmp/fwbuilder
$ rpm2cpio fwbuilder-5.1.0.3599-1.fc16.src.rpm | cpio -idvm
$ mv fwbuilder-5.1.0.3599.tar.gz ~/rpmbuild/SOURCES/
$ mv fwbuilder-5.1.0.3599.spec ~/rpmbuild/SPECS/

A patch file is required due to the following build error:

../fwbuilder/uint128.h: In member function 'std::string uint128::to_string() const':
../fwbuilder/uint128.h:469:95: warning: format '%lX' expects argument of type 'long unsigned int', but argument 3 has type 'long long unsigned int' [-Wformat]
../fwbuilder/uint128.h:469:95: warning: format '%lX' expects argument of type 'long unsigned int', but argument 4 has type 'long long unsigned int' [-Wformat]
../fwbuilder/uint128.h:471:58: warning: format '%lX' expects argument of type 'long unsigned int', but argument 3 has type 'long long unsigned int' [-Wformat]
In file included from ThreadTools.cpp:31:0:
../fwbuilder/ThreadTools.h:168:5: error: 'ssize_t' does not name a type
ThreadTools.cpp:182:82: error: no 'ssize_t libfwbuilder::TimeoutCounter::read(int, void*, size_t) const' member function declared in class 'libfwbuilder::TimeoutCounter'
make[4]: *** [.obj/ThreadTools.o] Error 1

The patch was built using the following diff (patch credit debian bug 674349)

$ diff -u src/libfwbuilder/src/fwbuilder/ThreadTools.h.orig src/libfwbuilder/src/fwbuilder/ThreadTools.h > ~/rpmbuild/SOURCES/fwbuilder-5.1.0.3599.ssize_t.patch

--- src/libfwbuilder/src/fwbuilder/ThreadTools.h.orig 2012-06-20 11:31:12.029364056 -0500
+++ src/libfwbuilder/src/fwbuilder/ThreadTools.h 2012-06-20 11:09:36.000000000 -0500
@@ -31,6 +31,7 @@
 
 #include  //for time_t definition
 #include 
+#include 
 
 #include 
 #include

Either run through the diff to generate the patch file or copy and paste the above diff into the new patch file ~/rpmbuild/SOURCES/fwbuilder-5.1.0.3599.ssize_t.patch Now edit the fwbuilder-5.1.0.3599.spec file to include the new patch file

--- fwbuilder-5.1.0.3599.spec.orig 2012-03-22 04:17:55.000000000 -0500
+++ fwbuilder-5.1.0.3599.spec 2012-06-20 11:51:09.901734348 -0500
@@ -20,6 +20,7 @@
 Group:          %{guigroup}
 Url:            http://www.fwbuilder.org/
 Source:         http://prdownloads.sourceforge.net/fwbuilder/%{name}-%{version}.tar.gz
+Patch1:         fwbuilder-5.1.0.3599.ssize_t.patch
 Packager:       Vadim Kurland 
 
 Buildroot:      %{_tmppath}/%{name}-%{version}-root
@@ -48,6 +49,7 @@
 
 %prep
 %setup 
+%patch1 -p0
 ./autogen.sh
 
 %build

Then run the build

$ cd ~/rpmbuild/SPECS
$ rpmbuild -ba fwbuilder-5.1.0.3599.spec

The result is the rpm containing

$ rpm -qpl ../RPMS/x86_64/fwbuilder-5.1.0.3599-1.fc17.uabeng.x86_64.rpm |more
/usr/bin/fwb_iosacl
/usr/bin/fwb_ipf
/usr/bin/fwb_ipfw
/usr/bin/fwb_ipt
/usr/bin/fwb_pf
/usr/bin/fwb_pix
/usr/bin/fwb_procurve_acl
/usr/bin/fwbedit
/usr/bin/fwbuilder
/usr/share/applications/fwbuilder.desktop
/usr/share/doc/fwbuilder-5.1.0.3599
...

Install the rpm using yum

$ sudo yum install ../RPMS/x86_64/fwbuilder-5.1.0.3599-1.fc17.uabeng.x86_64.rpm 
Loaded plugins: fs-snapshot, langpacks, presto, refresh-packagekit
Examining ../RPMS/x86_64/fwbuilder-5.1.0.3599-1.fc17.uabeng.x86_64.rpm: fwbuilder-5.1.0.3599-1.fc17.uabeng.x86_64
Marking ../RPMS/x86_64/fwbuilder-5.1.0.3599-1.fc17.uabeng.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package fwbuilder.x86_64 0:5.1.0.3599-1.fc17.uabeng will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================================================
 Package         Arch         Version                         Repository                                        Size
=====================================================================================================================
Installing:
 fwbuilder       x86_64       5.1.0.3599-1.fc17.uabeng        /fwbuilder-5.1.0.3599-1.fc17.uabeng.x86_64        36 M

Transaction Summary
=====================================================================================================================
Install  1 Package

Hope this helps other FC17 FWbuilder users out there!

Friday, June 1, 2012

HowTo: Converting EXT4 to BTRFS in Fedora 17

This HowTo documents the steps I used on a Fedora 17 x86_64 test machine to convert the root file system from Ext4 to Btrfs.

Anaconda (Fedora installer) doesn't support the creation or manipulation of Btrfs file systems in the graphical installer. Support for this is slated for Fedora 18, at which time, Btrfs may become the default file system!

It is possible, however, to create Btrfs file systems at install time via Kickstart.

The virtual machine was installed with Fedora 17 x86_64 using default partitioning (LVM and Ext4) and selecting the Desktop profile.

If possible, create the partitions without LVM. (see section at the bottom for an example of this and UUID)

In the examples, the root file system is a logical volume: /dev/mapper/vg_f17vm-lv_root

Following the install and the initial boot, I rebooted using the Fedora 17 x86_64 Live CD and chose the option to Try Fedora.

I initially tried to perform the conversion using the Fedora 17 DVD and rescue mode, however the btrfs-convert utility isn't included. The live CD, however, includes the Btrfs utilities.

Convert EXT4 root

All of the steps are performed using the terminal, so open a Gnome Terminal and SU to root:

$ su -

Run fsck on the file system

# fsck.ext4 -f /dev/mapper/vg_f17vm-lv_root

Convert from ext4 to btrfs. The metadata stage can take a while on a large populated file system (on an 1.5TB ext4 file system with approx 500GB of data, a good mix of large and small files, this step took about 1.5 hours)

# btrfs-convert /dev/mapper/vg_f17vm-lv_root

creating btrfs metadata.
creating ext2fs image file.
cleaning up system chunk.
conversion complete.

Mount the freshly converted btrfs root file system so that we can make some modifications (fstab, SELinux)

# mkdir /mnt/btrfs
# mount -t btrfs /dev/mapper/vg_f17vm-lv_root /mnt/btrfs

Edit fstab to change the root file system from ext4 to btrfs

# vi /mnt/btrfs/etc/fstab

#/dev/mapper/vg_f17vm-lv_root              ext4   defaults    1 1
/dev/mapper/vg_f17vm-lv_root              btrfs   defaults    1 1

Force the system to automatically relabel the SELinux context for the root file system during boot.

# touch /mnt/btrfs/.autorelabel

NOTE: In my test, the Fedora 17 system would hang with systemd library permissions errors prior to getting to the SELinux auto relabel step.

To work around this, I rebooted back into the Live CD and change the SELinux policy to "permissive" from "enforcing". During the next boot, the permission errors went by and then the SELinux autorelabel ran to completion.

# vi /mnt/btrfs/etc/selinux/config

#SELINUX=enforcing
SELINUX=permissive

The Btrfs conversion process created a subvolume of the old ext4 file system. The following commands mount the subvolume and file system, demonstrating the power of Btrfs.

# mkdir /mnt/{ext2_saved,ext4}
# mount -t btrfs -o subvol=ext2_saved /dev/mapper/vg_f17vm-lv_root /mnt/ext2_saved
# mount -t ext4 -o loop,ro /mnt/ext2_saved/image /mnt/ext4

The contents of the ext4 and btrfs mounts should be identical, sans the changes made above to fstab and config.

It's possible to roll back the conversion by first unmounting the file systems and then rolling it back using the -r switch

# umount /dev/mapper/vg_f17vm-lv_root
# btrfs-convert -r /dev/mapper/vg_f17vm-lv_root

If you decide to stay with btrfs, you can delete the subvolume snapshot of the original ext4 system

# btrfs subvolume delete /mnt/btrfs/ext2_saved

Reboot into the installed OS and the system should perform the relabel and then come up with your shiny new btrfs file system.

To make things really cool, install the yum-plugin-fs-snapshot Yum plugin. This plugin will create a btrfs subvolume of your root file system prior to installing updates. The end result, you can roll back from yum updates. Test that you can mount the snapshot of the ext4 file system

Once the system is up and running, don't forget to set the SELinux policy back to "enforcing"

Issues:

It appears that some of the systemd services attempt to start prior to the SELinux .autorelabel step. This results in permission errors that prevent the system from booting. Temporarily change the SELinux policy to "permissive" to work past this.
During boot, you may see errors logged by the systemd-fsck service failing for lv_root due to missing fsck.btrfs. I suspect that the btrfs-progs rpm failed to create a symlink of fsck.btrf pointing to /usr/sbin/btrfsck

systemd-fsck[858]: fsck: fsck.btrfs: not found
systemd-fsck[858]: fsck: error 2 while executing fsck.btrfs for /dev/mapper/vg_f17vm-lv_root

Convert Luks Encrypted EXT4 home

If your file system is encrypted with Luks, you'll have the additional step of decrypting the file system prior to running the conversion

# cryptsetup luksOpen /dev/mapper/vg_f17vm-lv_home luks_home
# fsck.ext4 -f /dev/mapper/luks_home
# btrfs-convert /dev/mapper/luks_home

creating btrfs metadata.
creating ext2fs image file.
cleaning up system chunk.
conversion complete.

Converting Non LVM Ext4 and UUID in /etc/fstab

If you don't use LVM (recommended based on everything I've ready, since it's not needed for btrfs), your /etc/fstab may mount the partitions using the UUID. After converting the file system to btrfs, the UUID will change.

These steps demonstrate how to convert the root file syste, a non LVM ext4 partition, /dev/sda2.

# fsck.ext4 -f /dev/sda2
# btrfs-convert /dev/sda2

creating btrfs metadata.
creating ext2fs image file.
cleaning up system chunk.
conversion complete.

Identify the UUID using blkid (can also look under /dev/disk/by-uuid

# blkid /dev/sda2

/dev/sda2: UUID="56d1e02b-3120-44c3-b8f8-d3289012e447" UUID_SUB="13f7b26b-42b1-41bb-a9dc-c928ad033f6d" TYPE="btrfs"

# mkdir /mnt/btrfs
# mount -t btrfs /dev/sda2 /mnt/btrfs

Edit fstab to change the root file system from ext4 to btrfs and replace the UUID with the new UUID

# vi /mnt/btrfs/etc/fstab

#UUID=e14bd8d6-ac8a-4f15-8169-f0a9bdf4b69e /                       ext4    defaults        1 1
UUID=56d1e02b-3120-44c3-b8f8-d3289012e447 /                       btrfs    defaults        1 1
UUID=f4cdf95d-66eb-4428-960d-23b289ffb63e /boot                   ext4    defaults        1 2

Chroot to the real root file system and update grub.cfg using grub2-mkconfig


# mount -o bind /dev /mnt/btrfs/dev
# mount -o bind /proc /mnt/btrfs/proc
# mount -o bind /sys /mnt/btrfs/sys
# mount /dev/sda1 /mnt/btrfs/boot

# chroot /mnt/btrfs

# grub2-mkconfig -o /boot/grub2/grub.cfg 

Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.4.3-1.fc17.x86_64
Found initrd image: /boot/initramfs-3.4.3-1.fc17.x86_64.img
Found linux image: /boot/vmlinuz-3.3.4-5.fc17.x86_64
Found initrd image: /boot/initramfs-3.3.4-5.fc17.x86_64.img
Warning: Please don't use old title `Fedora Linux, with Linux 3.3.4-5.fc17.x86_64' for GRUB_DEFAULT, use `Advanced options for Fedora Linux>Fedora Linux, with Linux 3.3.4-5.fc17.x86_64' (for versions before 2.00) or `gnulinux-advanced-/dev/sda2>gnulinux-3.3.4-5.fc17.x86_64-advanced-/dev/sda2' (for 2.00 or later)
done

Exit out of the chroot and unmount the file systems

# exit
# umount /mnt/btrfs/boot
# umount /mnt/btrfs/dev
# umount /mnt/btrfs/proc
# umount /mnt/btrfs/sys
# umount /mnt/btrfs




Reboot and you should be good to go. The selinux relabeling can take a while!
References:

https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3
https://btrfs.wiki.kernel.org/index.php/Getting_started
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sec-Plugin_Descriptions.html
http://fedoraproject.org/wiki/Anaconda/Kickstart#btrfs
http://www.h-online.com/open/news/item/Fedora-puts-back-Btrfs-deployment-yet-again-1436704.html
Good HowTo initially created for Fedora 13




Posted by


FlakRat




at
12:02 PM


3
comments














Email ThisBlogThis!Share to TwitterShare to Facebook



Labels:
btrfs,
ext4,
fedora,
fedora 17,
fs-snapshot,
fsck,
selinux,
yum


          

          
        
Friday, May 4, 2012

          
        




Script to Query Dell Warranty Status





I'm sure there are more elegant ways of obtaining the warranty status for your Dell hardware, but none-the-less, here's a Ruby script I wrote that does just that :-)



Script: query_dell_st.rb



The script currently has two modes, single lookup and batch lookup.



Single lookup takes two arguments:


--host HOSTNANE
--svctag SERVICE_TAG

Bulk requires only a single argument pointing to an input file containing a single entry "HOSTNAME SERVICE_TAG" per line for each device.


--file FILE

Example single lookup:



$ ./query_dell_st.rb --svctag GT7D8P1 --host lin-srv01
Host: lin-srv01
 Model: PowerEdge R410
 Service Tag: GT7D8P1
 Warranty Exp: 617 days left




Over the next few days I'll add more features (CSV and tab delimited output, for example) as well as address bugs and error handling.



Hopefully the script is useful to others.




Posted by


FlakRat




at
5:42 PM


1 comments














Email ThisBlogThis!Share to TwitterShare to Facebook



Labels:
centos,
dell,
fedora,
linux,
ruby







            
          

          
        
Friday, April 13, 2012

          
        




HowTo: Disable User List on RHEL6 / CentOS6 Login Window





Quick bit of code to disable the user list on the RHEL6 / CentOS6 login screen.




# gconftool-2 --direct --config-source=xml:readwrite:/etc/gconf/gconf.xml.defaults --type bool --set /apps/gdm/simple-greeter/disable_user_list true



Useful Gconf References:

http://live.gnome.org/GDM/2.22/Configuration
http://svn.gnome.org/viewvc/gdm/trunk/gui/simple-greeter/gdm-simple-greeter.schemas.in?view=markup





Posted by


FlakRat




at
1:53 PM


1 comments














Email ThisBlogThis!Share to TwitterShare to Facebook



Labels:
centos,
centos 6,
centos6,
fedora,
gconftool,
gnome,
greeter,
login,
rhel,
RHEL 6,
RHEL6




Older Posts

Home




Subscribe to:
Posts (Atom)

FlakRat's Rat House

Friday, April 19, 2013

How To Compile Gromacs 4.6.1 with OpenMPI 1.6.4 on CentOS

Intel Compilers

Build OpenMPI 1.6.4

Build Gromacs 4.6.1

Wednesday, February 27, 2013

Ubuntu 13.04 on Google Nexus 7

Wednesday, January 23, 2013

How To: Recursively Create Numeric Directories

First the Bash solution:

Now the Windows solution:

Wednesday, June 20, 2012

Build Firewall Builder 'FWbuilder' 5.1.0 for Fedora 17

Friday, June 1, 2012

HowTo: Converting EXT4 to BTRFS in Fedora 17

Convert EXT4 root

Convert Luks Encrypted EXT4 home

Converting Non LVM Ext4 and UUID in /etc/fstab

References:

Friday, May 4, 2012

Script to Query Dell Warranty Status

Friday, April 13, 2012

HowTo: Disable User List on RHEL6 / CentOS6 Login Window

Google+ Badge

Blog Archive

Get Fedora

Fuelly

About Me

Followers