Using Check_Openmanage with Check_MK via WATO

In an older post I described the steps to integrate check_openmanage Nagios plugin with check_mk. This approach required manually editing the etc/check_mk/main.mk file to configure the extra_nagios_conf and legacy_checks.

This updated guide uses the check_mk WATO (Web Administration Tool) to integrate the check_openmanage check using a feature called "Active Checks".

Here's the guide, hope it helps:

Environment:

• CentOS 6.5 x86_64 VMware ESXi hosted virtual machine
  
• OMD 1.10
  
• check_openmanage 3.7.11
  
• Check_mk 1.2.5i4 upgraded using these steps
  
• Dell servers are checked via check_openmanage using SNMP rather than MRPE (or NRPE)
  

Install Check_openmanage

Unless otherwise specified all paths are relative to the site owners home (ex: /opt/omd/sites/mysite)

1. Make sure your dell servers had the following SNMP packages installed prior to installing OMSA (if not, it's easy to 'yum remove srvadmin-\*' 'yum install srvadmin-all': net-snmp, net-snmp-libs, net-snmp-utils
   
   • Start the OMSA services 'srvadmin-services.sh start' and then check 'srvadmin-services.sh status' to verify that the snmpd component is running
     
   • Ensure that snmpd is running and configured
     
   • Configure the firewall to allow access from your OMD server to udp port 161
     
2. change users on your OMD server to the site user: $ su - mysite
   
3. Download the latest check_openmanage from http://folk.uio.no/trondham/software/check_openmanage.html to ~/tmp and extract
   
4. copy the check_openmanage script to local/lib/nagios/plugins (this defaults to $USER2$ in your commands)
   

   
   $ cp tmp/check_openmanage-3.7.11/check_openmanage local/lib/nagios/plugins/
   $ chmod +x local/lib/nagios/plugins/check_openmanage
   

5. copy the PNP4Nagios template
   

   
   $ cp tmp/check_openmanage-3.7.11/check_openmanage.php etc/pnp4nagios/templates/
   

6. Test check_openmanage to see that it can successfully query a node
   

   
   local/lib/nagios/plugins/check_openmanage -H dell-r720xd-01 -p -C MySecretCommunity
   
   OK - System: 'PowerEdge R720xd', SN: 'XXXXXX1', 24 GB ram (6 dimms), 2 logical drives, 14 physical drives|T0_System_Board_Inlet=21C;42;47 T1_System_Board_Exhaust=30C;70;75 T2_CPU1=48C;86;91 T3_CPU2=39C;86;91 W2_System_Board_Pwr_Consumption=126W;0;0 A0_PS1_Current_1=0.6A;0;0 A1_PS2_Current_2=0.2A;0;0 V25_PS1_Voltage_1=240V;0;0 V26_PS2_Voltage_2=240V;0;0 F0_System_Board_Fan1=2280rpm;0;0 F1_System_Board_Fan2=2280rpm;0;0 F2_System_Board_Fan3=2280rpm;0;0 F3_System_Board_Fan4=3000rpm;0;0 F4_System_Board_Fan5=3600rpm;0;0 F5_System_Board_Fan6=3480rpm;0;0
   
   

WATO Configuration

1. Create a Host Group by clicking Host Groups under WATO - Configuration, click New Group (click save when done):
   • Name: omsa
   • Alias: Dell OpenManage
2. Create a Host Tag by clicking Host Tags under WATO - Configuration, click New Tag Group (click save when done):
   • Internal ID: dellomsa
   • Topic: (leave empty)
   • Choices:
     • Tag ID: omsa
     • Description: Dell OpenManage
3. Create a Active Check by clicking Host & Service Parameter under WATO - Configuration, click Active Checks, click Classical active and passive Nagios checks (create a new one, click save when done):
   • Folder: Main Directory
   • Host Tags: Select Dell OpenManage is set
   • Service Description: check_openmanage
   • Commmand Line: $USER2$/check_openmanage -H $HOSTADDRESS$ -p -C MySecretCommunity
   • Service Description: check_openmanage
   • Check Perfomance Data
4. Add the omsa Host Tag to a host running OpenManage with SNMP configured by clicking Hosts under WATO - Configuration, and click the properties editor (pencil icon) for the host (click Save & go to Services when done):
   • Host tags: Dell OpenManage: check Dell OpenManage twice

On the Host services page you should see the new service at the bottom, example:

Custom checks (defined via rule)
Status  Checkplugin   Item              Service Description  Plugin output    
OK      custom        check_openmanage  check_openmanage     OK - System: 'PowerEdge R710', SN: 'XXXXXX1', 24 GB ram (6 dimms), 2 logical drives, 14 physical drives

Click Activate Missing services or Save manual check configuration. Activate the changes and you should start seeing the check within a few minutes and graphs after 10 minutes or so. Hope this helps, and comments are welcome.

Using check_dell_bladechassis with check_mk

This post builds off of a previous post that documented getting check_openmanage working with check_mk.

In this post we'll add check_dell_bladechassis to the mix to allow for monitoring of Dell M1000e blade chassis' (via the CMC management card).

This was done on the following system:

• CentOS 5.7 x86_64 KVM virtual machine
  
• OMD 0.50
  
• check_openmanage 3.7.3
  
• check_dell_bladechassis 1.0.0
  
• Dell M1000e chassis are queried via SNMP
  

Unless otherwise specified all paths are relative to the site owners home (ex: /opt/omd/sites/mysite) The check_openmanage code in this blog post is not necessary to get check_dell_bladechassis, I'm just including it to help tie this entry to the previous post.

1. Change users on your OMD server to the site user: $ su - mysite
   
2. Download the latest check_dell_bladechassis from http://folk.uio.no/trondham/software/check_dell_bladechassis.html to ~/tmp and extract
   
3. copy the check_dell_bladechassis script to local/lib/nagios/plugins (this defaults to $USER2$ in your commands)
   

   
   $ cp tmp/check_dell_bladechassis-1.0.0/check_dell_bladechassis local/lib/nagios/plugins/
   $ chmod +x local/lib/nagios/plugins/check_dell_bladechassis
   

4. copy the PNP4Nagios template
   

   
   $ cp tmp/check_dell_bladechassis-1.0.0/check_dell_bladechassis.php etc/pnp4nagios/templates/
   

5. Test check_dell_bladechassis to see that it can successfully query an M1000e CMC (I've inserted carriage returns in the output to make it more readable)
   

   
   local/lib/nagios/plugins/check_dell_bladechassis -H dell-m1000e-01 -p -C MySecretCommunity
   
   OK - System: 'PowerEdge M1000e', SN: 'XXXXXX', Firmware: '3.03', hardware working fine|
   'total_watt'=1500.000W;0;7928.000 'total_amp'=6.750A;0;0 'volt_ps1'=239.500V;0;0 
   'volt_ps2'=242.750V;0;0 'volt_ps3'=242.750V;0;0 'volt_ps4'=241.750V;0;0 'volt_ps5'=241.750V;0;0 
   'volt_ps6'=242.750V;0;0 'amp_ps1'=1.688A;0;0 'amp_ps2'=1.641A;0;0 'amp_ps3'=0.188A;0;0 
   'amp_ps4'=1.516A;0;0 'amp_ps5'=1.500A;0;0 'amp_ps6'=0.219A;0;0
   
   

6. Edit the main.mk file to define the command, etc... (the perfdata_format and monitoring_host I got from a previous emailer to the list, not sure if they are needed)
   

   
   all_hosts = [
    'dell-m1000e-01|snmp|m1000e|nonpub',
    'dell-r710-01|linsrv|kvm|omsa|nonpub',
    'dell-2950-01|linsrv|omsa|nonpub',
    'hp-srv-01|winsrv|smb', ]
   
   # Are you using PNP4Nagios and MRPE checks? This will make PNP
   # choose the correct template for standard Nagios checks:
   perfdata_format = "pnp"
   #set the monitoring host
   monitoring_host = "nagios"
   
   # SNMP Community
   snmp_default_community = "someCommunityRO"
   
   snmp_communities = [
     ( "MySecretCommunity", ["nonpub"], ALL_HOSTS ),
   ]
   
   extra_nagios_conf += r"""
   
   # ARG1: community string
   define command {
       command_name    check_openmanage
       command_line    $USER2$/check_openmanage -H $HOSTADDRESS$ -p -C $ARG1$
   }
   
   define command {
       command_name    check_dell_bladechassis
       command_line    $USER2$/check_dell_bladechassis -H $HOSTADDRESS$ -p -C $ARG1$
   }
   
   """
   
   legacy_checks = [
     # On all hosts with the tag 'omsa' check Dell OpenManage for status 
     # service description "Dell OMSA", process performance data
     ( ( "check_openmanage!MySecretCommunity", "Dell OMSA", True), [ "omsa" ], ALL_HOSTS ),
     # similar for m1000e
     ( ( "check_dell_bladechassis!MySecretCommunity", "Dell Blade Chassis", True), [ "m1000e" ], ALL_HOSTS ),
   
   ]
   
   

7. That should be it, reinventory your M1000e and reload
   

   
   $ check_mk -II dell-m1000e-01
   $ check_mk -O
   
   

8. The php code has a bug that can be fixed using the below patch (see the first comment for details)
   

   
   --- a/check_dell_bladechassis.php 2009-08-04 07:00:15.000000000 -0500
   +++ b/check_dell_bladechassis.php 2011-12-21 14:44:25.488132187 -0600
   @@ -41,7 +41,7 @@
     
     $opt[$count] = "--slope-mode --vertical-label \"$vlabel\" --title \"$def_title: $title\" ";
     
   -        $def[$count] .= "DEF:var$i=$rrdfile:$DS[$i]:AVERAGE " ;
   +        $def[$count] = "DEF:var$i=$rrdfile:$DS[$i]:AVERAGE " ;
            $def[$count] .= "AREA:var$i#$PWRcolor:\"$NAME[$i]\" " ;
            $def[$count] .= "LINE:var$i#000000: " ;
    
   @@ -62,7 +62,7 @@
     
     $opt[$count] = "-X0 --lower-limit 0 --slope-mode --vertical-label \"$vlabel\" --title \"$def_title: $title\" ";
     
   -        $def[$count] .= "DEF:var$i=$rrdfile:$DS[$i]:AVERAGE " ;
   +        $def[$count] = "DEF:var$i=$rrdfile:$DS[$i]:AVERAGE " ;
            $def[$count] .= "AREA:var$i#$AMPcolor:\"$NAME[$i]\" " ;
            $def[$count] .= "LINE:var$i#000000: " ;
    
   @@ -75,6 +75,7 @@
        if(preg_match('/^volt_/',$NAME[$i])){
     if ($visited_volt == 0) {
         ++$count;
   +     $def[$count] = '';
         $visited_volt = 1;
     }
     
   @@ -87,6 +88,7 @@
     
     $def[$count] .= "DEF:var$i=$rrdfile:$DS[$i]:AVERAGE " ;
     $def[$count] .= "LINE:var$i#".$colors[$v++].":\"$NAME[$i]\" " ;
   +
     $def[$count] .= "GPRINT:var$i:LAST:\"%3.2lf $UNIT[$i] last \" ";
     $def[$count] .= "GPRINT:var$i:MAX:\"%3.2lf $UNIT[$i] max \" ";
     $def[$count] .= "GPRINT:var$i:AVERAGE:\"%3.2lf $UNIT[$i] avg \\n\" ";
   @@ -96,6 +98,7 @@
        if(preg_match('/^amp_/',$NAME[$i])){
     if ($visited_amp == 0) {
         ++$count;
   +     $def[$count] = '';
         $visited_amp = 1;
     }
    
   

Hope this helps, and comments are welcome.

Using check_openmanage with check_mk

Here's my guide to installing check_openmanage in an OMD site, in case it helps anyone:

This was done on the following system:

• CentOS 5.5 x86_64 KVM virtual machine
  
• OMD 0.47.20110302
  
• check_openmanage 3.6.5
  
• Dell servers are checked via check_openmanage using SNMP rather than MRPE (or NRPE)
  

Unless otherwise specified all paths are relative to the site owners home (ex: /opt/omd/sites/mysite)

1. Make sure your dell servers had the following SNMP packages installed prior to installing OMSA (if not, it's easy to 'yum remove srvadmin-\*' 'yum install srvadmin-all': net-snmp, net-snmp-libs, net-snmp-utils
   
   • Start the OMSA services 'srvadmin-services.sh start' and then check 'srvadmin-services.sh status' to verify that the snmpd component is running
     
   • Ensure that snmpd is running and configured
     
   • Configure the firewall to allow access from your OMD server to udp port 161
     
2. change users on your OMD server to the site user: $ su - mysite
   
3. Download the latest check_openmanage from http://folk.uio.no/trondham/software/check_openmanage.html to ~/tmp and extract
   
4. copy the check_openmanage script to local/lib/nagios/plugins (this defaults to $USER2$ in your commands)
   

   
   $ cp tmp/check_openmanage-3.6.5/check_openmanage local/lib/nagios/plugins/
   $ chmod +x local/lib/nagios/plugins/check_openmanage
   

5. copy the PNP4Nagios template
   

   
   $ cp tmp/check_openmanage-3.6.5/check_openmanage.php etc/pnp4nagios/templates/
   

6. If you are running CentOS 5.5/RHEL 5.5 or earlier (it's unclear whether or not this will be an issue in EL5.6), and you want performance graphs, you'll need to edit the check_openmanage.php template (see this bug: https://bugs.op5.com/bug_view_advanced_page.php?bug_id=4008), comment out the original condition and replace:
   

   
   $ vi etc/pnp4nagios/templates/check_openmanage.php
   
   ##    if(preg_match('/^enclosure_(?.+?)_temp_\d+$/', $NAME[$i], $matches)
   ##       || preg_match('/^e(?.+?)t\d+$/', $NAME[$i], $matches)){
   # This is the fixed line for CentOS 5.5 and prior
        if(preg_match('/^enclosure_(.+?)_temp_\d+$/', $NAME[$i], $matches)){
   

7. Test check_openmanage to see that it can successfully query a node (ack, I need to update my driver)
   

   
   local/lib/nagios/plugins/check_openmanage -H dell-r710-01 -p -C MySecretCommunity
   
   Controller 1 [SAS 5/E Adapter]: Driver '3.04.13rh' is out of date|fan_0_system_board_fan_1_rpm=3600;0;0 fan_1_system_board_fan_3_rpm=3600;0;0 fan_2_system_board_fan_4_rpm=3600;0;0 fan_3_system_board_fan_5_rpm=3600;0;0 fan_4_system_board_fan_2_rpm=3600;0;0 pwr_mon_0_ps_1_current=0.6;0;0 pwr_mon_1_ps_2_current=0.4;0;0 pwr_mon_2_system_board_system_level=182;0;0 temp_0_system_board_ambient=21;42;47
   
   

8. Edit the main.mk file to add tags to the OMSA hosts and the check command (the perfdata_format and monitoring_host I got from a previous emailer to the list, not sure if they are needed)
   

   
   all_hosts = [ 'dell-r710-01|linsrv|kvm|omsa|nonpub', 'dell-2950-01|linsrv|omsa|nonpub', 'hp-srv-01|winsrv|smb', ]
   
   # Are you using PNP4Nagios and MRPE checks? This will make PNP
   # choose the correct template for standard Nagios checks:
   perfdata_format = "pnp"
   #set the monitoring host
   monitoring_host = "nagios"
   
   # SNMP Community
   snmp_default_community = "someCommunityRO"
   
   snmp_communities = [
     ( "MySecretCommunity", ["nonpub"], ALL_HOSTS ),
   ]
   
   # other main.mk stuff
   
   extra_nagios_conf += r"""
   
   # ARG1: community string
   define command {
       command_name    check_openmanage
       command_line    $USER2$/check_openmanage -H $HOSTADDRESS$ -p -C $ARG1$
   }
   
   """
   
   legacy_checks = [
     # On all hosts with the tag 'omsa' check Dell OpenManage for status 
     # service description "Dell OMSA", process performance data
     ( ( "check_openmanage!MySecretCommunity", "Dell OMSA", True), [ "omsa" ], ALL_HOSTS ),
   ]
   

9. That should be it, simply reload and your new check should start working for all hosts tagged with 'omsa'
   

   
   $ check_mk -O
   
   

To make it cleaner, the legacy_check should be able to determine the community string based on the settings in snmp_default_community and snmp_communities

I've only been testing check_mk for a few days now and am not sure how to do that. (suggestions?)

Hope this helps, and comments are welcome.