Thursday, March 6, 2008

System Administration with Puppet

I manage quite few Linux workstations (mostly Red Hat EL4 and EL5, CentOS) and found that my home grown scripting approach to maintaining these systems was becoming increasingly difficult.

One obvious problem with using a mountain of scripts to maintain systems, what happens when I want to go on vacation and have to leave the Windows guy in charge? I needed a system that would make it easy for a temporary admin (or new team member) to take the reigns.

A couple recent issues of Linux Magazine (a great publication by the way) reviewed CFengine and Puppet for systems management. I had known that there were open source solutions out there, but had not spent the time to do the research, preferring to handle the latest challenge by writing another script.

The articles finally convinced me to do further research. I ultimately chose Puppet for several reasons, including:
  • Written in Ruby
  • Very easy to get a base system up and running
  • Helpful community
  • Active development
I really like the fact that the configuration files use Ruby syntax, so by configuring Puppet, I'm also learning something about Ruby. I'm a Perl guy, but have been interested in learning Ruby for system administration. Now I have the excuse :-)

Example Syntax
A short example of the power of Puppet.

class baseclass {
   service { "sshd":
      enable => true,
      ensure => running,

   file { "/etc/shadow":
      owner => "root",
      group => "root",
      mode => 400,

Puppet will ensure that nodes that include this class (baseclass) have the sshd service set to startup on boot and verify that it is running, starting it if necessary. Puppet will also verify the owner, group and permissions of the /etc/shadow file, if they don't match our definition Puppet will make the necessary changes.

These definitions will work regardless of the target operating system (Red Hat EL, Fedora, SuSE, Sun Solaris, BSD, whatever). That's very appealing to administrators because they don't have to script for each unique environment, Puppet handles it in the background.

Puppet uses another Ruby based utility called 'facter' that returns a whole slew of data about the node (architecture, hostname, fqdn, ipaddress, kernelrelease, ...). You can use these facter defined variables in your configurations. For example, Puppet provides for basic 'if' and 'case' statements to allow you to define conditions, for example, only include the class '64bitonly' if the architecure (returned by facter) is x86_64:

case $architecture {
   x86_64: { include 64bitonly }
   default: { }

Useful Puppet Links

Saturday, March 1, 2008

LaCrosse WS2308 Weather Station

I installed a LaCrosse WS2308 weather station with the idea of having my CentOS 5 Linux server uploading the data from the WS2308 to Weather Underground ( If you are a techie, you've gotta see the coolness of logging into Weather Underground and seeing live weather data at your house. My station's link:

The WS2308 comes with Windows software, not much help on CentOS (maybe it'd work with Wine?) and connects to the PC via serial cable. LaCrosse is kind enough to offer a free high quality serial to USB adapter available upon request. Luckily, my server is old enough to have serial ports (although I did order the free USB adapter, I couldn't get it to work. dmesg showed it identified but I couldn't communicate with the device).

I tried several software packages and settled on a perl script called This script uses the module Device::LaCrosse::WS23xx to query the WS2308. I decided to update the script to extract additional information as well as a simultaneous upload to a MySQL database.

Here's the updated script, unfortunately wraps long lines so for the script to work you'll need to account for that):

# Weather Underground PWS Upload Script for Lacrosse WS23xx weather stations over serial device
# Contributed by Kenneth Brown 12/2007
# Use settings below -- must edit idline, freq (to match cron frequency), and possibly serial
# Example cron entry to fire off uploads (if you install this script into /usr/local/bin)
# 0,10,20,30,40,50 * * * * /usr/local/bin/ >> wu-log.log
# usage - cron this for at least a one minute run, my serial port takes between 5-10 seconds to report the values on average.#
# perl/c + modules installed on FC7 x86 w/ standard development libraries, tested on usb-serial connection
# GPL - original code, K Brown 11/30/07
# Updated by FlakRat
# - Changed from collecting local time to gmtime, that's apparently what WeatherUnderground is expecting
# - Added a $debug option that will provide more verbose logging
# - Modified verbose output to mask the password
# - Added 2 settings that will output a mysql insert statement to stdout, and a second that will insert the data
# into a mysql database. This has the advantage of importing into a local db and uploading to WeatherU during
# the same command execution

### these modules are on :

use LWP::UserAgent;
use HTTP::Request;
use Device::LaCrosse::WS23xx;
use Mysql;

### define wu sites

my $wuurl = ""; ## standard update ( deprecated)
my $rturl = ""; ## real-time update

### specify these items ->
my $idline = "ID=&PASSWORD="; ### plaintext password over http - insecure !
my $freq = 300; ### update frequency in seconds
#my $serial = "/dev/ttyUSB0"; ### serial port, in this case a keyspan serial->usb
my $serial = "/dev/ttyS0"; ### serial port, in this case a keyspan serial->serial

my $nosend = 0; ### 1 for testing, don't upload
my $verbose = 0; ### 1 for testing cmd line, or write to stdout for sending to a log
my $debug = 0; ### 1 for debugging cmd line more than just verbose, or write to stdout for sending to a log
my $mysqlout = 0; ### 1 to output to stdout a SQL INSERT statement, this can be piped into a file for future insertion into MySQL
my $mysqlinsert = 1; ### 1 to insert the data into MySQL

### MySQL Info
my $host = "localhost";
my $database = "weather";
my $tablename = "weather_history";
my $mysqluser = "";
my $mysqlpass = "";

### get the data
my $ws = Device::LaCrosse::WS23xx->new($serial)
or die "Cannot communicate with $serial: $!\n";
#my $ws = Device::LaCrosse::WS23xx->new($serial,trace => '/tmp/wu-upload.trace')
# or die "Cannot communicate with $serial: $!\n";

die "wireless connection failure\n" unless ( $ws->get("Outdoor_Temperature","F"));

#@then = localtime($ws->get("Date/Time_last_record_datetime"));
#@then = localtime(time);
# localtime returns CST on my system, we want UTC
@then = gmtime(time);

$dateutc=sprintf("%4d-%02d-%02d %02d:%02d:%02d",$then[5]+1900,$then[4]+1,$then[3],$then[2],$then[1],$then[0]);
print "dateutc: $dateutc\n" if $debug;
$dateutc = "&dateutc=$dateutc";
$winddir = $ws->get("Wind_Direction");
print "winddir: $winddir\n" if $debug;
$windspeedmph = $ws->get("Wind_Speed","mph");
#$windspeedmph = $ws->get("Wind_Speed");
print "windspeedmph: $windspeedmph\n" if $debug;
#$windgustmph = $ws->get("Wind_Gust","mph");
#print "windgustmph: $windgustmph\n" if $debug;
$tempf = $ws->get("Outdoor_Temperature","F");
print "tempf: $tempf\n" if $debug;
$rainin = $ws->get("Rain_1hour","in");
print "rainin: $rainin\n" if $debug;
$dailyrainin = $ws->get("Rain_24hour","in");
print "dailyrainin: $dailyrainin\n" if $debug;
$baromin = $ws->get("Absolute_Pressure","inHg");
print "baromin: $baromin\n" if $debug;
$dewptf = $ws->get("Dewpoint","F");
print "dewptf: $dewptf\n" if $debug;
$humidity = $ws->get("Outdoor_Humidity");
print "humidity: $humidity\n" if $debug;

my $timestamp_gmt = sprintf("%4d%02d%02d%02d%02d%02d",$then[5]+1900,$then[4]+1,$then[3],$then[2],$then[1],$then[0]);
my $rec_date = sprintf("%4d-%02d-%02d",$then[5]+1900,$then[4]+1,$then[3]);
my $rec_time = sprintf("%02d:%02d:%02d",$then[2],$then[1],$then[0]);
my $temp_in = $ws->get("Indoor_Temperature","F");
my $temp_out = $ws->get("Outdoor_Temperature","F");
my $dewpoint = $ws->get("Dewpoint","F");
my $rel_hum_in = $ws->get("Indoor_Humidity");
my $rel_hum_out = $ws->get("Outdoor_Humidity");
my $windspeed = $ws->get("Wind_Speed","mph");
my $wind_angle = "";
my $wind_direction = $ws->get("Wind_Direction");
my $wind_chill = $ws->get("Windchill","F");
my $rain_1h = $ws->get("Rain_1hour","in");
my $rain_24h = $ws->get("Rain_24hour","in");
my $rain_total = $ws->get("Rain_Total","in");
my $abs_pressure = $ws->get("Absolute_Pressure","inHg");
my $rel_pressure = $ws->get("Relative_Pressure","inHg");
my $tendency = $ws->get("Tendency");
my $forecast = $ws->get("Forecast");
my $myquery =
"INSERT INTO $tablename " .
"(timestamp_gmt,rec_date,rec_time,temp_in,temp_out,dewpoint,rel_hum_in,rel_hum_out,windspeed,wind_direction,wind_chill,rain_1h,rain_24h,rain_total,abs_pressure,rel_pressure,tendency,forecast) " .
print "$myquery\n" if $mysqlout;

if ($mysqlinsert) {
$connect = Mysql->connect($host, $database, $mysqluser, $mysqlpass);


$execute = $connect->query($myquery);

my $url =
$idline .
"&dateutc=$dateutc" .
"&winddir=$winddir" .
"&windspeedmph=$windspeedmph" .
"&tempf=$tempf" .
"&baromin=$baromin" .
"&dewptf=$dewptf" .
"&humidity=$humidity" .

##### change above realtime var if you change the url, only the new one supports it

my $now = `date`;
my $urlmask = "$url";
$urlmask =~ s/PASSWORD=[a-zA-Z0-9]*&/PASSWORD=xxxxxxxxxx&/;
print "$now: send $urlmask\n" if $verbose;

unless ( $nosend ) {
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => "$rturl$url");
$resp = $ua->simple_request($req);
$response = $resp->content;

my $now = `date`;
# print "$response\n" if $verbose;
print "$now: $response\n" if $verbose;
die "$response\n" unless ( $response =~ /success/i ) ;


###### data section contains constants,units for input and output



get($;$) func param 1

Indoor_Temperature C
Min_Indoor_Temperature C
Max_Indoor_Temperature C
Min_Indoor_Temperature_datetime time_t
Max_Indoor_Temperature_datetime time_t
Low_Alarm_Indoor_Temperature C
High_Alarm_Indoor_Temperature C
Outdoor_Temperature C
Min_Outdoor_Temperature C
Max_Outdoor_Temperature C
Min_Outdoor_Temperature_datetime time_t
Max_Outdoor_Temperature_datetime time_t
Low_Alarm_Outdoor_Temperature C
High_Alarm_Outdoor_Temperature C
Windchill C
Min_Windchill C
Max_Windchill C
Min_Windchill_datetime time_t
Max_Windchill_datetime time_t
Low_Alarm_Windchill C
High_Alarm_Windchill C
Dewpoint C
Min_Dewpoint C
Max_Dewpoint C
Min_Dewpoint_datetime time_t
Max_Dewpoint_datetime time_t
Low_Alarm_Dewpoint C
High_Alarm_Dewpoint C
Indoor_Humidity %
Min_Indoor_Humidity %
Max_Indoor_Humidity %
Min_Indoor_Humidity_datetime time_t
Max_Indoor_Humidity_datetime time_t
Low_Alarm_Indoor_Humidity %
High_Alarm_Indoor_Humidity %
Outdoor_Humidity %
Min_Outdoor_Humidity %
Max_Outdoor_Humidity %
Min_Outdoor_Humidity_datetime time_t
Max_Outdoor_Humidity_datetime time_t
Low_Alarm_Outdoor_Humidity %
High_Alarm_Outdoor_Humidity %
Rain_24hour mm
Max_Rain_24hour mm
Max_Rain_24hour_datetime time_t
Rain_1hour mm
Max_Rain_1hour mm
Max_Rain_1hour_datetime time_t
Rain_Total mm
Rain_Total_datetime time_t
Min__wind m/s
Max__wind m/s
Min_Date/Time_wind_datetime time_t
Max_Date/Time_wind_datetime time_t
Wind_Speed m/s
Wind_Direction degrees
Low_wind_alarm_setting m/s
High_wind_alarm_setting m/s
Countdown_time_to_next_datBinary seconds
Absolute_Pressure hPa
Relative_Pressure hPa
Pressure_Correction hPa
Min_Absolute_Pressure hPa
Min_Relative_Pressure hPa
Max_Absolute_Pressure hPa
Max_Relative_Pressure hPa
Min_Pressure_datetime time_t
Max_Pressure_datetime time_t
Low_Alarm_Pressure hPa
High_Alarm_Pressure hPa
History_saving_interval minutes
Countdown_to_next_saving minutes
Date/Time_last_record_datetime time_t

Only a few reasonable UNIT conversions are available

get($;$) function param 2:

From To
---- --
hPa inHh, mmHg
m/s kph, mph, kt
mm in

Here is the URL used in the uploading (if you go here without parameters
you will get a brief usage): ## real time updates ( preferred )

Here is the usage that will be displayed with blank parameters:
action [action=updateraw]
ID [ID as registered by]
PASSWORD [PASSWORD registered with this ID]
dateutc - [YYYY-MM-DD HH:MM:SS (mysql format)]
winddir - [0-360]
windspeedmph - [mph]
windgustmph - [windgustmph ]
humidity - [%]
tempf - [temperature F]
rainin - [rain in (hourly)] -- the accumulated rainfall in the past 60 mins
dailyrainin - [rain so far today in localtime]
baromin - [barom in]
dewptf- [dewpoint F]
weather - [text] -- metar style (+RA)
clouds - [text] -- SKC, FEW, SCT, BKN, OVC
soiltempf - [temp F]
soilmoisture - [%]
leafwetness - [%]
solarradiation - [MJ/m^2]
UV - [index]
visibility - [nm]
softwaretype - [text] ie: vws or weatherdisplay
#realtime http only
rtfreq - frequency in seconds for nominal update

The MySQL table is defined below. I create a new database called "weather" and create the weather_history table in the new database.

DROP TABLE IF EXISTS `weather_history`;
CREATE TABLE `weather_history` (
`timestamp_gmt` bigint(14) NOT NULL default '0',
`rec_date` date NOT NULL default '0000-00-00',
`rec_time` time NOT NULL default '00:00:00',
`temp_in` decimal(3,1) NOT NULL default '0.0',
`temp_out` decimal(3,1) NOT NULL default '0.0',
`dewpoint` decimal(3,1) NOT NULL default '0.0',
`rel_hum_in` tinyint(3) NOT NULL default '0',
`rel_hum_out` tinyint(3) NOT NULL default '0',
`windspeed` decimal(3,1) NOT NULL default '0.0',
`wind_angle` decimal(3,1) NOT NULL default '0.0',
`wind_direction` char(3) NOT NULL default '',
`wind_chill` decimal(3,1) NOT NULL default '0.0',
`rain_1h` decimal(3,1) NOT NULL default '0.0',
`rain_24h` decimal(3,1) NOT NULL default '0.0',
`rain_total` decimal(4,1) NOT NULL default '0.0',
`abs_pressure` decimal(4,1) NOT NULL default '0.0',
`rel_pressure` decimal(4,1) NOT NULL default '0.0',
`tendency` varchar(7) NOT NULL default '',
`forecast` varchar(6) NOT NULL default '',
UNIQUE KEY `timestamp_gmt` (`timestamp_gmt`)

Fedora 8 on Dell XPS m1330

Fedora 8 needs some essential tweaking to get it running the way I like it on my Dell XPS m1330. My laptop has the Intel 4965 AGN wifi card and the nVidia 8400M GS video card, both of which work right out of the box. I had originally gone cheap and got the Dell wifi card (a re-branded Brodcomm) and it required a hack to make it work (ndiswrapper) which I wasn't interested in:

Yum Repositories
Install the livna and Adobe repositories, the protect_base package etc...

$ sudo rpm -i
$ sudo rpm -ivh

$ sudo yum install yum-protectbase
$ sudo perl -pi -e "s/(\[.*\])/\1\nprotect=yes/" /etc/yum.repos.d/{fedora*,livna*}

$ sudo yum install yum-presto yumex k3b

For some strange reason, when ~/.kde is created (not sure if this is done during account creation or when k3b is installed), ownership on ~/.kde/shared is root:root. This prevents the k3b cd burner program from starting:
sudo chown -R flakrat:flakrat ~/.kde/shared

Nvidia Driver
Install the nVidia driver from the livna repository:

$ sudo yum install kmod-nvidia xorg-x11-drv-nvidia-libs-32bit

And add the following section (in bold) to /etc/X11/xorg.conf

Section "Files"
ModulePath "/usr/lib/xorg/modules/extensions/nvidia"
ModulePath "/usr/lib/xorg/modules"

Section "Module"
Load "glx"
Load "extmod"

Section "ServerFlags"
Option "AIGLX" "on"

Synaptics Touch Pad
The Synaptics touch pad default settings have the annoying horizontal and vertical scroll areas enabled. Firefox makes this extra painful because any time you touch that area, Firefox wants to browse forward or back in history. The touch pad also registers taps while typing.

Edit /etc/X11/xorg.conf and modify the InputeDevice section as follows (more options are described via 'man synaptics':
Section "InputDevice"
Identifier "Synaptics"
Driver "synaptics"
Option "Device" "/dev/input/mice"
Option "Protocol" "auto-dev"
Option "Emulate3Buttons" "yes"
Option "SHMConfig" "on"
Option "VertScrollDelta" "0"
Option "HorizScrollDelta" "0"


Now disable touchpad tapping while typing. This is done by adding a startup command.
  1. Open the sessions manager: System -> Preferences -> Personal -> Sessions
  2. With the Startup Programs tab open, click Add
  3. Type in the following:
  • Name: Disable Touchpad tapping while typing
  • Command: syndaemon -t -i 1 -d
  • Comment: Disables Touchpad tapping while typing
The -t only disables tapping and scrolling while typing, not mouse movements while the '-i 1' sets the idle time to 1 second and -d runs it as a daemon.

Fedora 8 doesn't start NetworkManager by default, so this needs to be set to automatically started. The gnome-keyring will ask for your keyring password each time you attempt to connect to a network, this can be changed by using pam_keyring.

Also, uncheck "Activate on Boot" for both eth0 and wlan0 in system-config-network to prevent eth0 and wlan0 looking for ip addresses on boot. After you login, NetworkManager will handle this (resulting in much quicker bootups).

$ sudo /sbin/chkconfig NetworkManager on
$ sudo /sbin/service NetworkManager start

Install pam_keyring

$ sudo yum install pam_keyring

And edit the pamt.d/gdm file to look like (bold lines are the 2 that were added)

$ sudo vim /etc/pam.d/gdm

auth [success=done ignore=ignore default=bad]
auth required
auth optional try_first_pass
auth include system-auth
auth optional
account required
account include system-auth
password include system-auth
session required close
session include system-auth
session required
session optional
session required open
session optional force revoke
session required
session optional auto_start
session optional

You might need to delete your keyring file and have it recreated (make sure to use the same password you use to login):
rm -rf ~/.gnome2/keyrings

Firefox Essential Addons
1. Tab Mix Plus (
2. Forecastfox Enhanced (
3. Download Statusbar (
4. Goggle Toolbar (
MSN Messenger
There's a great MSN Messenger clone for Linux called AMSN:

$ sudo yum install amsn

Open Office
Fedora 8 comes with Open Office 2, however I've found that at least one important Calc (Excel) feature is missing in the Fedora packaged version; the ability to select a sequence and then drag the sequence to other cells to have it continue. For example, say I want a column with compute-a, compute-b, compute-c ---> compute-z. In Excel you can create the first two entries and then drag the selection down and Excel will continue the sequence.

With Fedora 8's Open Office, this only appears to work for simple numeric sequences. Uninstalling this version and reinstalling from the official Open Office rpms enables this feature. Some on the Open Office message boards suggest that Red Hat may purposely disable certain features over fears of software patents. I'm not sure if this is true, but the feature certainly works with the official Open Office packages.

Download the OOo_2.4.0_LinuxIntel_install_wJRE_en-US.tar.gz file from and extract it. Run the setup script and you'll be Open Office'n in no time.
Occasionally, I run across an application that will not work with Wine or natively in Fedora. One example, the CSTV website requires Internet Explorer and other components of Windows. This calls for a virtual machine. I use VMware Virtual Server and ESX Server at work, but figured at home I'd try out VirtualBox (recently purchased by Sun):

1. Download and install the rpm
sudo rpm -i VirtualBox-1.6.0_30421_fedora8-1.i586.rpm
2. Add your userid to the vboxusers group
sudo /usr/sbin/usermod -a -G vboxusers flakrat
3. Start the app, Applications -> System Tools -> Sun xVM VirtualBox (this doesn't appear on the menu until you reboot or possibly logout login)