Cisco Prime Nerk 43 User Guide

							 
C-9
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Device Unreachable on Link Down Event
Figure C-7 illustrates the lab setup for a Device Unreachable on a Link Down event.
Figure C-7 Lab Setup for Device Unreachable on a Link Down Event
Description of Fault Scenario in the Network
The S.2/3 port of PE-East connected to the S.1/2 port of the CE-5 device (also called L1 link) is set to 
administrative status down. This effectively takes the L1 link down.
Related Faults
The CE-5 device is managed from this link with no backup. With the L1 link down, the CE-5 device is 
unreachable from the management subnet.
Prime Network Failure Processing
Event Identification
The following service alarms are generated by the system:
[Device Unreachable, CE-5] event.
The device unreachability event means that no other information can be collected from this device 
by the VNE.
[Link Down Due to Admin Down, PE-East < > CE-5] event.
The Link Down event is issued by the PE-East VNE (active) as a result of the link down negotiation 
process. 
						

							 
C-10
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Noncorrelating Events
The noncorrelating event is:
[Link Down Due to Admin Down, PE-East < > CE-5]
This event opens a new ticket in the gateway.
The L1 Link Down event is configured to not correlate to other events. This is logical because the edge 
VNEs identify the Link Down events as [Link Down Due to Admin Down] events. This implies that the 
VNEs know the root cause of the event already, based on the administrator’s configurations. The [Link 
Down Due to Admin Down] events reach the northbound interface immediately after the links’ new 
statuses are discovered by Prime Network and after the link down negotiation methods are completed.
Possible Root Cause
1.Prime Network waits two minutes.
2.After two minutes, the [Device Unreachable, CE-5] event triggers the CE-5 VNE to initiate an 
IP-based flow to the management subnet:
Flow Path: CE-5 > PE-East > management subnet
Root Cause Selection
For the event [Device Unreachable, CE-5]:
Collected Events: [Link Down Due to Admin Down, PE-East < > CE-5].
NoteOther possible events are also collected, such as Interface Status Down events.
Root Cause: Correlates to [Link Down Due to Admin Down, PE-East < > CE-5].
Figure C-8 displays the events identified by the system in this scenario.
Figure C-8 Device Unreachable on Link Down
NoteIn Figure C-8, port E.0/3 should read S.2/3, and E.0/2 should read S.1/2.
Clearing Phase
When the PE-East port S.2/3 (L1 link) comes up again, the CE-5 reachability from the management 
subnet also returns. The CE-5 starts responding to polling requests from the CE-5 VNE. The device is 
declared reachable, thus clearing the Device Unreachable alarm. The L1 Link Down is cleared when the 
PE-East device indicates that the status of the connected port has changed to up. 
Link down due to admin downPE-East-IOU-161#0:Ethernet0/3ce-5-IOU-161#0:Etherenet0/2
370858
PE-East-IOU-161 VRF vfrB IP:Ethernet0/3 : 169.254.161.223
PE-East-IOU-161 VRF vfrB IP:Ethernet0/3
PE-East-IOU-161 VRF vfrB IP:Ethernet0/3
PE-East-IOU-161 VRF vfrB IP:Ethernet0/3
ce-5-IOU-161
Event Correlation Hierarchy Location
OSPF neighbor down syslog
Interface status down
Link down syslog
Line down syslog
Device unreachable 
						

							 
C-11
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Multiroute Correlation Scenarios
Figure C-9 displays the lab multiroute configuration setup between P-South, P-North, and P-West 
devices. The OSPF cost is the same along the path from P-South and P-North whether or not it goes via 
P-West; that is, P-South and P-North connect along two paths with equal cost.
Figure C-9 Lab Multiroute Configuration Setup Between P-South, P-North and P-West
Description of a Fault Scenario in the Network
In this example, the P-North, P-South link (also known as L2) goes down in a multiroute segment 
between P-South and P-North. After approximately one minute, another link, L1 (PE-East, P-North), 
also goes down. Both links go down administratively, the first from the P-North device and the second 
from the PE-East devices’ ports. 
Related Faults
Almost all devices are unreachable from the management subnet. This discussion focuses on CE-1 
unreachability (see Figure C-1).
NoteSyslogs and traps corresponding to network faults are also reported. Additional related faults might also 
be reported, but are not described in this topic.
Prime Network Failure Processing
Event Identification
The following service alarms are generated by the system:
[Device Unreachable, CE-1] event. 
The device unreachability event means that no other information can be collected from this device 
by the VNE.
[Link Down Due to Admin Down, P-North < > PE-East] event. 
						

							 
C-12
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
The Link Down event is issued by the PE-East VNE (active) as a result of the link down negotiation 
process.
[Link Down Due to Admin Down, P-North < > P-South] event.
The Link Down event is issued by the P-North VNE as a result of the link down negotiation process.
Noncorrelating Events
[Link Down Due to Admin Down, P-North < > PE-East] opens a new ticket in the gateway.
[Link Down Due to Admin Down, P-North < > P-South] opens a new ticket in the gateway.
For more information, see Noncorrelating Events, page C-10.
Possible Root Cause
1.Prime Network waits two minutes.
2.After two minutes, the [Device Unreachable, CE-1] event triggers the CE-1 VNE to initiate an 
IP-based flow to the management IP subnet:
Flow Path: CE-1 > Cloud > PE-South > P-South > P-North > PE-East > management subnet
Flow Path: CE-1 > Cloud > PE-South > P-South > P-West > P-North > PE-East > management 
subnet
Root Cause Selection
For the event [Device unreachable, CE-1]:
For the flow path CE-1 > Cloud > PE-South > P-South > P-North > PE-East > management subnet:
–Collected Events: [Link Down Due to Admin Down, P-North < > PE-East] and
[Link Down Due to Admin Down, P-South > P-North]. 
NoteOther possible events are also collected, such as Interface Status Down events.
–Root Cause—Correlates to:
[Link down due to admin down, P-SouthS.1/0 > P-North S.1/0 < > PE-East S.2/2] and
[Link down due to admin down, P-NorthS.1/3 > PE-East S.2/2]
For the Flow Path 
CE-1 > Cloud > PE-South > P-South > P-West > P-North > PE-East > management subnet:
Root Cause: Correlates to [Link Down Due to Admin Down, P-North S.1/0 < > PE-East S.2/2]
NoteThe CE-1’s VNE root cause selection method identifies the Device Unreachable event’s root cause on 
the L1 Link Down event. According to the logic, when two flows split and result in two sets of possible 
root cause events, sets that are supersets of others (depending on whether both flows end at the same 
location) are removed. Sets that are not removed are united into one set containing all events. This 
implies that, in this scenario, the set that includes both links is removed because it is a superset of the 
set that contains only the L1 link.
NoteAll devices that are unreachable correlate their unreachability events to the L1 link as expected. 
						

							 
C-13
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Figure C-10 displays the events identified by the system in this scenario (L1).
Figure C-10 Multiroute Scenario—L1
Figure C-11 displays the events identified by the system in this scenario (L2).
Link down due to admin downP-North-IOU-161#0:Serial1/3PE-East-IOU-161#0:Serial2/2
PE-East-IOU-161 IP:Serial2/2
PE-East-IOU-161 IP:Serial2/2 PE-East-IOU-161 IP:Serial2/2
PE-East-IOU-161
PE-East-IOU-161
PE-East-IOU-161 : 169.254.161.224
PE-East-IOU-161
PE-East-IOU-161 : 169.254.161.216
ce-3-IOU-161
ce-1-IOU-161
ce-2-IOU-161
PE-South-IOU-161
RR1-IOU-161
P-West-IOU-161
P-South-IOU-161
P-North-IOU-161
PE-East-IOU-161
PE-East-IOU-161 RR2-IOU-161
370859
Event Correlation Hierarchy Location
Interface status down
Link down syslog
LDP neighbor down syslog
BGP neighbor down syslog
BGP neighbour loss
BGP neighbor down syslog
Device unreachable
Device unreachable
Device unreachable
Device unreachable
Device unreachable
Device unreachable
Device unreachable
Device unreachable
LDP neighbor down
Line down syslog
Device unreachable
LDP neighbor down
LDP neighbor down syslog 
						

							 
C-14
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Figure C-11 Multiroute Scenario—L2
Clearing Phase
Enabling the L1 link makes the CE-1 device reachable from the management subnet IP address, thereby 
clearing the Device Unreachable event of the CE-1 device. When the L1 link’s new status is discovered 
by Prime Network, the PE-East device eventually initiates a Link Up event for this link. When the 
administrator enables the Layer 2 link and Prime Network discovers this change, the Link Down event 
is cleared by its matching Link Up event.
BGP Neighbor Loss Correlation Scenarios
The VNE models the BGP connection between routers and actively monitors its state. BGP neighbor loss 
events are generated from both sides of the connection only when connectivity is lost, and when the other 
side of the link is unmanaged.
The correlation engine identifies various faults that affect the BGP connection and reports them as the 
root cause for the BGP Neighbor Loss alarm; for example, Link Down, CPU Overutilized, and Link Data 
Loss.
Link down due to admin downP-North-IOU-161#0:Serial1/0P-South-IOU-161#0:Serial1/0
P-North-IOU-161 IP:Serial1/0
P-North-IOU-161 IP:Serial1/0 P-North-IOU-161 IP:Serial1/0 : 169.254.161.214
P-North-IOU-161 IP:Serial1/0
P-South-IOU-161 IP:Serial1/0 : 169.254.161.213
P-South-IOU-161 IP:Serial1/0
P-South-IOU-161 IP:Serial1/0
P-North-IOU-161
P-North-IOU-161
P-South-IOU-161
P-South-IOU-161
370860
Event Correlation Hierarchy Location
Interface status down
OSPF neighbor down syslog
OSPF neighbor down syslog
Interface status down
Line down syslog
LDP neighbor down
LDP neighbor down syslog
LDP neighbor down
LDP neighbor down syslog
Line down syslog
Link down syslog 
						

							 
C-15
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Figure C-12 Lab Setup for BGP Neighbor Loss Correlation Scenarios
NoteIn Figure C-12 the link between P-West and PE-North-West is not real and merely emphasizes how 
PE-North-West is connected in the network.
There are two main scenarios that might lead to a BGP neighbor loss event:
BGP neighbor loss due to a Link Down (or an equivalent port down).
BGP neighbor loss due to BGP Process Down or device down. 
						

							 
C-16
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
BGP Neighbor Loss Due to Port Down
Figure C-13 displays the BGP neighbor loss due to port down scenario.
Figure C-13 BGP Neighbor Loss Due to Physical Port Down (P-West > PE-North-West)
Description of Fault Scenario in the Network
In Figure C-13 the BGP neighbor loss occurs due to a physical port down (in P-West that connects to 
PE-North-West). The relevant devices are PE-North-West, RR2, P-North and P-West.
Related Faults
Port on P-West that is connected to the PE-North-West goes down.
BGP neighbor, on RR2, to PE-North-West changes state from Established to Idle.
NoteSyslogs and traps corresponding to network faults are also reported. Additional related faults might also 
be reported, but are not included in this discussion.
Prime Network Failure Processing
Event Identification
The following service alarms are generated by the system:
[BGP Neighbor Loss, RR2] event.
Since the VNE that monitors each PE or RR holds records of the entire device’s BGP information, 
the change in the BGP table is identified by the VNE and causes it to send this event. 
						

							 
C-17
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Possible Root Cause
1.Prime Network waits two minutes. For more information, see How Prime Network Correlates 
Incoming Events, page 10-4.
2.After two minutes, the [BGP Neighbor Loss, RR2] event triggers the VNE to initiate an IP-based 
flow to the destination IP of its lost BGP neighbor (PE-North-West):
Flow Path: RR2 > P-North > P-West > P-West port is connected to PE-North-West (which is 
unmanaged), and is in a down state.
Root Cause Selection
For the event [BGP Neighbor Loss, RR2]:
Collected Events: [Port Down, P-West].
Root Cause: Correlates to [Port Down, P-West].
Figure C-14 displays the events identified by the system in this scenario.
Figure C-14 BGP Neighbor Loss Due to Physical Port Down
Clearing Phase
When a Port Up event is detected by the system for the same port that was detected as the root cause for 
the BGP Neighbor Loss event, the alarm is cleared. The ticket is cleared (colored green) when all the 
alarms in the ticket have been cleared.
Port down
P-West-IOU-176#0:Serial...
P-West-IOU-176 IP:Serial...
P-West-IOU-176 IP:Serial... P-West-IOU-176 IP:Serial...
P-West-IOU-176 IP:Serial...
P-West-IOU-176 IP:Serial...
P-West-IOU-176
P-West-IOU-176
RR2-IOU-176 : 169.254....
RR2-IOU-176
RR1-IOU-176 : 169.254....
RR1-IOU-176
370861
Event Correlation Hierarchy
Location
OSPF neighbor down syslog
Interface status down
Line down trap
LDP neighbor down
LDP neighbor down syslog
BGP neighbor down syslog
BGP neighbour loss
BGP neighbor down syslog
BGP neighbour lossLine down syslog
Link down syslog 
						

							 
C-18
Cisco Prime Network 4.3.2 User Guide
Appendix C      Event Correlation Examples
  Correlation Scenarios
Figure C-15 displays the up event that clears all the down events identified by the system.
Figure C-15 BGP Neighbor Up Event that Clears All the Down Events
Variation
In a BGP process down scenario, the BGP Process Down event is identified by the system in addition to 
the BGP Neighbor Loss event.
As illustrated in Figure C-16, the BGP Process Down event causes several events (the BGP Neighbor 
Loss event cannot be seen). The relevant devices are RR2 (BGP Process Down, marked in red) and 
PE-North-West (marked as unmanaged).
Port up
P-West-IOU-176#0:Serial...
P-West-IOU-176 IP:Serial...
P-West-IOU-176 IP:Serial... P-West-IOU-176 IP:Serial...
P-West-IOU-176 IP:Serial...
P-West-IOU-176 IP:Serial...
P-West-IOU-176
P-West-IOU-176
RR2-IOU-176 : 169.254....
RR2-IOU-176
RR1-IOU-176 : 169.254....
RR1-IOU-176
370862
Event Correlation Hierarchy
Location
OSPF neighbor up syslog
Interface status up
Line down trap
LDP neighbor up
LDP neighbor up syslog
BGP neighbor up syslog
BGP neighbour found
BGP neighbor up syslog
BGP neighbour foundLine up syslog
Link up syslog