Detect Network Failure using EEM and Traceroute

Posted by
Facebooktwitterredditpinterestlinkedintumblr

Cisco IOS Embedded Event Manager (EEM) is very useful automation tool particularly in detecting network failures. In this post, I’m going to show you how to leverage EEM and traceroute to detect this issue. The network topology we are using is in the figure.

Let say for example, R7 and R8 are the routers you managed. You happened to know the provider’s managed router R3 has two connections (primary and backup) and you would like to determine (by some sort of an syslog message) when the provider has issue on their primary connection.

EEM traceroute

The configuration snippet is used to achieve this goal. The EEM runs the traceroute sourcing from Loopback2 every minute. The if-statement logic is used to detect whether 23.0.0.1 hop is found or not on the traceroute output and it generates a syslog message. If the CPE is configured to send all its syslog messages to a management station, the syslog message can be used to trigger email alert to network admins. Other way is you can use EEM to send email alert if you don’t a management station. Check my post here on how to do this Cisco IOS EEM: Monitor ISDN Layer 2 Status.

event manager applet CHECK-TRACEROUTE-HOP
 event timer cron name DAILY-1MIN cron-entry "* * * * *"
 action 0.1 cli command "enable"
 action 1.1 set DESTINATION "40.1.1.1"
 action 1.2 set HOP-FOUND "23.0.0.1"
 action 1.3 cli command "trace $DESTINATION source loo2"
 action 1.4 regexp "23.0.0.1" "$_cli_result"
 action 1.5 if $_regexp_result eq "1"
 action 1.6  syslog msg "Found hop 23.0.0.1"
 action 1.7 else
 action 1.8  syslog msg "Did not find hop"
 action 1.9 end

Configuration verification.

*Sep 11 09:09:00.043: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : CTL : cli_open called.
*Sep 11 09:09:00.055: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : R8>
*Sep 11 09:09:00.055: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : IN  : R8>enable
*Sep 11 09:09:00.095: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : R8#
*Sep 11 09:09:00.095: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : IN  : R8#trace 40.1.1.1 source loo2
*Sep 11 09:09:00.679: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : Type escape sequence to abort.
*Sep 11 09:09:00.679: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : Tracing the route to 40.1.1.1
*Sep 11 09:09:00.679: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : VRF info: (vrf in name/id, vrf out name/id)
*Sep 11 09:09:00.679: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT :   1 38.0.0.1 24 msec 8 msec 8 msec
*Sep 11 09:09:00.683: %
R8#HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT :   2 23.0.0.1 16 msec 28 msec 28 msec
*Sep 11 09:09:00.683: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT :   3 12.0.0.1 40 msec 40 msec 52 msec
*Sep 11 09:09:00.683: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT :   4 14.0.0.2 12 msec 60 msec 56 msec
*Sep 11 09:09:00.683: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : R8#
*Sep 11 09:09:00.683: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP: Found hop 23.0.0.1
*Sep 11 09:09:00.683: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : CTL : cli_close called.
*Sep 11 09:09:00.687: 
*Sep 11 09:09:00.687: tty is now going through its death sequence

*Sep 11 09:10:00.047: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : CTL : cli_open called.
*Sep 11 09:10:00.055: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : R8>
*Sep 11 09:10:00.055: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : IN  : R8>enable
*Sep 11 09:10:00.067: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : R8#
*Sep 11 09:10:00.067: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : IN  : R8#trace 40.1.1.1 source loo2
*Sep 11 09:10:00.331: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : Type escape sequence to abort.
*Sep 11 09:10:00.331: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : Tracing the route to 40.1.1.1
*Sep 11 09:10:00.331: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : VRF info: (vrf in name/id, vrf out name/id)
*Sep 11 09:10:00.331: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT :   1 38.0.0.1 20 msec 20 msec 12 msec
*Sep 11 09:10:00.331:
R8# %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT :   2 13.0.0.1 20 msec 16 msec 20 msec
*Sep 11 09:10:00.331: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT :   3 14.0.0.2 16 msec 20 msec 28 msec
*Sep 11 09:10:00.331: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : OUT : R8#
*Sep 11 09:10:00.331: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP: Did not find hop
*Sep 11 09:10:00.335: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOP : DEBUG(cli_lib) : : CTL : cli_close called.
*Sep 11 09:10:00.347: 
*Sep 11 09:10:00.347: tty is now going through its death sequence

Leave a Reply

Your email address will not be published. Required fields are marked *

*