Cisco - EEM Scripts Examples (TSHOOT)

EEM is a very useful tool to troubleshoot occasional, sporadic CPU spikes that are short-lived and difficult to troubleshoot manually with the command line interface. This is an example of CPU spikes:

Switch#show process cpu history
<snip>
    11111822511   11 111277711111 124111  11 1211111112161116
    143342171209994090111878458239607111981270283361362429475
100
 90
 80      *               ***
 70      *               ***                                *
 60      *               ***                            *   *
 50      *  *            ***        *                   *   *
 40      *  *            ***        *                   *   *
 30      * **            ***        *                   *   *
 20      ****           **** **   ***         **  *  ** ** **
 10 *********************************************************
   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
             0    5    0    5    0    5    0    5    0    5    0    5    0
                   CPU% per hour (last 72 hours)
                  * = maximum CPU%   # = average CPU%

This section includes several examples of the use of EEM scripts to monitor CPU utilization. Catalyst 2960 and 3750 switches allow EEM to use non-volatile RAM (NVRAM); Catalyst 4500 switches allow EEM to write to bootflash; and Catalyst 6500 switches allow EEM to use disk0 and sup-bootdisk.

Email Alerts

This script emails an alert when CPU utilization goes above 50 percent. The body of the email is the output of theshow process cpu sorted command.

event manager applet highcpu
  event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.3.1 get-type exact entry-op ge 
  entry-val 50 poll-interval 0.5
  action 1.0 cli command "enable"
  action 2.0 cli command "show proc cpu sorted"
  action 3.0 mail server "192.168.1.1" to "user-to@domain.com" from "user-from@domain.com" 
  subject "High CPU Alert" body "$_cli_result"

The definitions of italicized variables are:

highcpu - name of the event manager applet/script

1.3.6.1.4.1.9.9.109.1.1.1.1.3.1 - Object identifier (OID) for polling the total CPU utilization of the route processor (RP)

50 - CPU utilization that triggers the script
poll-interval 0.5 - Frequency (every 0.5 seconds) the script monitors the CPU

192.169.1.1 - IP of the mail server

Append Output to Local File

This script appends required outputs to a file in the local file system. Replace file system with the appropriate file system on the switch.

event manager scheduler script thread class default number 1 
event manager applet High_CPU
event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.3.1 get-type exact entry-op ge  
    entry-val 50 poll-interval 0.5
  action 0.0 syslog msg "High CPU DETECTED. Please wait - logging Information  
    to file system:high_cpu.txt"
  action 0.1 cli command "enable"
  action 0.2 cli command "show clock | append file system:high_cpu.txt"
  action 1.2 cli command "term length 0"
  action 1.3 cli command "show process cpu sorted | append file system:high_cpu.txt"
  action 1.4 cli command "show log | append file system:high_cpu.txt"
  action 1.5 cli command "show interfaces | append file system:high_cpu.txt"
  action 1.6 cli command "term length 24"

Append Output to Local File and Remove Script

This script appends the output of the show process cpu sorted command to a file in the local file system, then removes itself once completed. Replace file system with the appropriate file system on the switch.

event manager scheduler script thread class default number 1 
event manager applet High_CPU
event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.3.1 get-type exact entry-op gt 
    entry-val 50 poll-interval 0.5
  action 0.0 syslog msg "High CPU DETECTED. Please wait - logging Information  
    to flash:high_cpu.txt"
  action 0.1 cli command "enable"
  action 0.2 cli command "term exec prompt timestamp"
  action 1.3 cli command "show process cpu sorted | append file system:high_cpu.txt"
  action 1.4 cli command "show process cpu sorted | append file system:high_cpu.txt"
  action 1.4 cli command "show process cpu sorted | append file system:high_cpu.txt"
  action 5.1 syslog msg "Finished logging information to file system:high_cpu.txt..."
  action 5.1 syslog msg "Self-removing applet from configuration..."
  action 5.2 cli command "term no exec prompt timestamp"
  action 9.1 cli command "configure terminal"
  action 9.2 cli command "no event manager applet High_CPU"
  action 9.3 cli command "end"

Collect Output and Write to Local File

This script uses a syslog-based trigger in order to run and collect required outputs and write those outputs to the local file system. Replace file system with the appropriate file system on the switch.

process cpu threshold type total rising 70 interval 15 
event manager applet DETECT_CPU
event syslog pattern ".*SYS-1-CPURISINGTHRESHOLD.*" 
  action 1 cli command "en" 
  action 2 cli command "show clock | append file system:cpuinfo" 
  action 3 cli command "show proc cpu sort | append file system:cpuinfo" 
  action 4 cli command "show line | append file system:cpuinfo"

Monitor CPU Utilization on Modular IOS

The Cisco EEM can also be used to monitor CPU utilization on modular IOS. Because of the differences in how the CPU is monitored on modular IOS, you can use the Simple Network Management Protocol (SNMP) OID (1.3.6.1.4.1.9.9.109.1.1.1.1.3.1) in order to check CPU utilization by the IOS base process.

This script uses the OID as a trigger and writes required outputs to the local file system. Replace file system with the appropriate file system on the switch.

event manager scheduler script thread class default number 1
event manager applet High_CPU
event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.10.1  get-type exact entry-op ge  
    entry-val 50 poll-interval 0.5 
  action 0.0 syslog msg "High CPU DETECTED. Please wait - logging Information  
    to file system:high_cpu.txt"
  action 0.1 cli command "enable"
  action 0.2 cli command "show clock | append file system:high_cpu.txt"
  action 1.2 cli command "term length 0"
  action 1.3 cli command "show process cpu sorted | append file system:high_cpu.txt"
  action 1.4 cli command "show log | append file system:high_cpu.txt"
  action 1.5 cli command "show interfaces | append file system:high_cpu.txt"
  action 1.6 cli command "term length 24"

Remove Script

Enter this command in order to remove an EEM script:

Switch(config)#no event manager applet applet 

Based On: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/116141-trouble-eem-scripts-00.html
Contributed by Cisco Engineers: Shashank Singh and Saurav Lahiri