Opsview Knowledge Center

Network Analyzer

Learn how to use Network Analyzer

This section explains the concept of Network Analyzer; including what NetFlow, sFlow and jFlow data is, how it is sent back into Opsview Monitor, and finally how the gathered data can be interpreted and analyzed.

Overview

Opsview Monitor's Network Analyzer is an add-on module for Opsview Monitor Pro, Enterprise and MSP customers which enables the collection and analysis of flow-enabled network devices, such as NetFlow from Cisco routers, sFlow from HP Switches and more.

The main benefit of flow protocols such as NetFlow and sFlow is that they allow you to look 'inside' the connection to see not only 'That link is 95% utilized', but to understand why ' i.e. is a User downloading large files continuously, is a router misconfigured, etc.

Network Analyzer currently supports the three main flow protocols:

  • NetFlow
  • jFlow
  • sFlow

Network devices that support one of the three protocols above can be configured to send data 'back' to Opsview Monitor, so that it can be stored, analyzed and displayed. The configuration of each protocol varies from vendor to vendor, however on a Cisco router the main part of the required configuration is:

ip flow-export source Ethernet0/0 ip flow-export destination 192.168.11.11 9997

You then have to configure 'ip flow ingress' on each router sub interface you wish to monitor the traffic of. To view the configuration of the Cisco router, run the command 'show ip flow export', as shown below:

2611#show ip flow export
Flow export v1 is enabled for main cache
Exporting flows to 192.168.11.11 (9997) 192.168.15.23 (9997)
Exporting using source interface Ethernet0/0
Version 1 flow records
Cache for destination-prefix aggregation:
Flow export is disabled
35687335 flows exported in 4525686 udp datagrams
0 flows failed due to lack of export packet
4525685 export packets were sent up to process level
0 export packets were dropped due to no fib
0 export packets were dropped due to adjacency issues
0 export packets were dropped due to fragmentation failures
0 export packets were dropped due to encapsulation fixup failures

For more information on configuring Cisco devices to export NetFlow data, see the link
here. For information on configuring NetFlow/jFlow/sFlow for your network device, please contact your network device vendor.

As you will notice, we have configured the router to send flow data to a specific destination. This destination must be either an Opsview Monitor master or slave server, which is known as a Flow collector ' i.e. something that collects inbound flow packets and stores them. The device sending the flow data is known as a source, i.e. the source of the data.

In the example above, you can see that we have three routers that are exporting flow data; one is sending flow data to a slave server, and the remaining two are sending flow data to the master (representing two separate locations). The routers are labeled 'flow sources', and the Opsview Monitor servers are 'flow collectors'.

Flow data can be sent to either a master server or a slave server, as all data will be synced back to the master server via SSH to be stored at /var/opt/opsview/netflow. We recommend you have a disk partition for this directory to allow you to separate Flow data from the other operating system usage on /var.

Note: You may need to open ports through your firewall to allow the network devices to pass data to Opsview Monitor. Check your device configuration on which ports to open.

Adding a Flow Collector

Before adding a Flow Collector, you must first ensure that any Opsview Monitor server you wish to have Flow data sent into, has the 'Application ' Opsview NetFlow Common' Host Template applied. Also, ensure that the Opsview Monitor master server has the 'Application ' Opsview NetFlow Master' Host template applied.

To add a Flow Collector, navigate to 'Network Analyzer Settings' within the 'Settings' tab of the overlay menu, as shown below:

Once within the page, you will be able to view the Flow Collectors configured (by default this section will be empty):

To add a new Flow Collector, click on the 'Add New' button in the top left. This will load a modal window as shown below:

The modal window contains four fields:

  • Name: The name of the Flow Collector, i.e. 'Master', 'UK', etc.
  • NetFlow Port: The port to which NetFlow data should be sent to; this can be modified.
  • sFlow Port: The port to which sFlow data (and also jFlow) data should be sent to. This is also User- modifiable.
  • Monitoring Server: Choose the monitoring server which will act as the newly configured Flow Collector. A monitoring server can only be configured as a Flow Collector once, i.e. if you have one slave server and one master, you can have a maximum of two Flow Collectors.

Once you have configured your Flow Collector, click 'Submit Changes' and the new Flow Collector will be added, as shown below: You will notice that whilst the name and ports can be modified, the Monitoring Server field cannot be edited once saved.

Troubleshooting

Syncing Data Between Nodes

If you get this message when editing or creating a monitoring server, you will need to manually synchronise historical Flow data:

As NetFlow data is stored between multiple nodes in a slave cluster, it is possible that the data is not consistent between different nodes. For instance, this could occur if you have just added a new node to a cluster, or there was a node failure. To synchronise the NetFlow data, we assume the node with the canonical data is called cnode and the other node is called othernode. Firstly, ensure othernode is running correctly and do an Opsview reload. Then:

othernode$ cd /var/opt/opsview/netflow/data/
# Ensure that the network device is sending NetFlow data and that data is being written and # check that SOURCEID/YYYY/MM/DD/ has a timestamp for time just passed
cnode$ su - nagios
cnode$ rsync -avz --exclude="nfcapd.current.*" /var/opt/opsview/netflow/data/ othernode:/var/opt/opsview/netflow/data/
cnode$ # Wait for file rotation, ie a new timestamp file appears
cnode$ rsync -avz --exclude="nfcapd.current.*" /var/opt/opsview/netflow/data/ othernode:/var/opt/opsview/netflow/data/

There are two rsyncs, to ensure the last timestamp is synchronised. All time afterwards will be kept the same due to the NetFlow source sending to all nodes.

Note: Before you remove the Network Analyzer feature, please delete any sources and collectors, otherwise you will not be able to edit some information on the host or slave.

Adding a Source

Now that you have added your Flow Collectors, you can configure the Flow Sources, a Flow Source being a Host that is sending Flow data inbound into an Opsview Monitor server.

To add a Flow Source, edit a collector and navigate to the 'Sources' tab:

In the example screenshot above, we have added a Flow Collector but it has no Flow Sources yet. To add a new Flow Source, click on the 'Add new' button which will load a new section at the bottom of the screen:

In the example above, we have added our Flow Source, 'Cisco2611', and its IP address has been determined via lookup. If the IP address cannot be looked up, then the field will display an error as shown below:

This error is displayed as Opsview Monitor needs to know the IP address of the Host in order to map the received Flow data (which contains a 'From:' field containing an IP address) to the Host, i.e. 'rutland.opsera.com == 192.168.15.12'. To add the IP, click on 'Override IP Address?' and manually enter the IP address.

Once the correct Host and IP address has been entered, ensure you have selected 'Active' ' otherwise the Flow Source wont be displayed within the Network Analyzer Dashlets.

Click 'update' to save any changes you have made to the individual Flow Source, and then finally click 'Submit Changes' to close the window and save the newly added Flow Sources.

A reload is then required to apply the changes to both the Flow Collector and the newly-added Flow Sources.

Note: Before you remove the Network Analyzer feature, please delete any sources and collectors, otherwise you will not be able to edit some information on the host or slave.

Configuration Options

Within the 'Network Analyzer Settings' window, there is a 'gear cog' icon which loads the global settings for the Network Analyzer module:

Clicking on this 'Settings gear cog' icon will load a modal window as shown below:

This window allows you to change the retention period of data received from Flow Sources, along with an indicator showing the average data per day used per Flow Source, and the estimated storage required based on the values entered.

The average data/day is based on data over the last week. If there is less than a week's worth of data, the actual duration of data will be displayed in parenthesis, eg (three hours), so that you have an idea of how representative the value will be.

There are two different retention periods:

  • Summary Retention Period: How long the Opsview Monitor system should retain summary data, i.e. rounded up, summarised values as opposed to the per second/minute data files initially received. This data is stored within the 'opsview_netflow_stats' table in the Runtime database on the Opsview Monitor master server.
  • Full Data Retention Period: How long the Opsview Monitor system should retain the 'raw' data, i.e. the files within /var/opt/opsview/netflow that contain the per second/minute data.

Analyzing the Data

Now that you have configured your Flow Collector(s) and its Flow Sources, you can begin to analyze the received data. Analysis of Flow data is done via 'My Dashboards', using one or more of the seven available 'Network Analyzer' Dashlets, located within the 'Settings' Dashlet drawer, as shown below:

The seven available Dashlets are:

  • Sources Summary
  • Sources History
  • Top 10 Host Transmitters
  • Top 10 Host Receivers
  • Top 10 Port Transmitters
  • Top 10 Port Receivers
  • Top 10 Transfers

In the following sections we will cover each of the Dashlets, covering how to configure the Dashlet and what data is returned.

Sources Summary

The Sources Summary Dashlet lists all the configured Flow Sources and the average amount of data received from them as well as the last update time:

The configuration options available for this Dashlet can be seen below:

In the above configuration modal window, you can choose which sources are displayed within the Sources Summary Dashlet (either individually select the sources, or select 'All sources' to display all sources within the Opsview Monitor system).

There is also a 'Duration' field within the 'Options' drawer which allows you to determine the 'average bytes' period, i.e. 'Last 5 mins', 'Last 15 mins', 'Last 30 mins' or 'Last 1 Hr'.

Sources History

The Sources History Dashlet allows you to view the data transferred through each Flow Source, and investigate specific points in time:

To investigate a time, i.e. perhaps a spike in the throughput, hover your mouse over the time period which will display a button labelled 'Investigate', as below:

When 'Investigate' is clicked, a modal investigation window will load as below:

This investigate window will display the data as it was at the point in time selected, i.e. it will display the 'top 10 transfers' that were occurring at the selected time.

You can also choose to change the duration from one of the following options: 'Last 1 min', 'Last 2 mins', 'Last 5 mins', 'Last 10 mins' and 'Last 20 mins'. When the duration is changed, the buttons to the right will change, i.e. '-10' and '+10' if the duration is set to 'Last 10 mins'. These buttons, along with the '-1' and '+1' buttons, allow you to step through the period in time ' i.e. 'What does it look like 10 minutes from now?'.

This is great troubleshooting tool, as you can step through minute by minute to see what was happening on the network, and at which specific time it began to get overloaded (for example).

The configuration for the 'Sources History' Dashlet contains two drawers; 'Filter by Sources' and 'Graph Settings', as below:

The 'Filter by Sources' drawer allows you to choose which Flow Sources are displayed on the Dashlet. The 'Graph Settings' drawer, displayed above, allows you to fine tune how the graph is shown:

Data Type: Bytes, Packets or Flows.
Protocol: All, UDP, TCP, ICMP, Other
Duration: 1 hour, 3 hours, 6 hours, 12 hours, 1 day, 2 days, 3 days, 5 days, 10 days, 30 days
Chart style: Line, Area, Stack
Line Thickness: Various options

There is also the option to bind the 'Y-Axis to 0'.

Finally, you can choose to 'zoom' into a specific section of the Sources History Dashlet by left-clicking and dragging your mouse over a specific area, which will redraw the Dashlet to focus just on the selected time period, as below:

Top 10 Host Transmitters

The 'Top 10 Host Transmitters' Dashlet will display the 'Top 10 Talkers' in terms of Bytes and packets transmitted per host, on a collector basis:

For example, in the above Dashlet we have chosen a collector, 'My Collector', which contains just the one Flow Source. The Dashlet therefore is showing that the Host '192.168.11.74' is transmitting 55.3% of all bytes that are passing through the Flow Source (A Cisco router, in this example).

The configuration options for this Dashlet can be seen below:

You must choose a Flow Collector first, i.e. an Opsview Monitor master or slave server. After choosing the Flow Collector, the Flow Sources list will populate with all the Flow Sources created on the chosen Flow Collector. You can then choose to use data from all Flow Sources, or select Flow Sources one by one.

Finally, the 'Options' drawer allows you to define the 'Duration:', i.e. the time period the data is gathered over. Options include 'Last 1 Min', 'Last 2 Mins', 'Last 5 Mins' and 'Last 10 Mins'.

Top 10 Host Receivers

The 'Top 10 Host Receivers' Dashlet will display the 'Top 10 Downloaders' in terms of Bytes and packets received per host, on a collector basis:

For example, in the above Dashlet we have chosen a collector, 'My Collector', which contains just the one Flow Source. The Dashlet therefore is showing that the Host 'Cisco2611' is receiving 100% of all the bytes sent via the Flow Source (A Cisco router, in this example).

The configuration options for this Dashlet can be seen below:

You must choose a Flow Collector first, i.e. an Opsview Monitor master or slave server. After choosing the Flow Collector, the Flow Sources list will populate with all the Flow Sources created on the chosen Flow Collector. You can then choose to use data from all Flow Sources, or select Flow Sources one by one.

Finally, the 'Options' drawer allows you to define the 'Duration:', i.e. the time period the data is gathered over. Options include 'Last 1 Min', 'Last 2 Mins', 'Last 5 Mins' and 'Last 10 Mins'

Top 10 Port Transmitters

The 'Top 10 Port Transmitters' Dashlet will display the 'Top 10 Ports' in terms of Bytes and packets transmitted (i.e. data is being sent from Port 123), on a collector basis:

For example, in the above Dashlet we have chosen a collector, 'My Collector', which contains just the one Flow Source. The Dashlet therefore is showing that the port 'ICMP' is transmitting 88.4% of all the bytes sent via the Flow Source (A Cisco router, in this example).

The configuration options for this Dashlet can be seen below:

You must choose a Flow Collector first, i.e. an Opsview Monitor master or slave server. After choosing the Flow Collector, the Flow Sources list will populate with all the Flow Sources created on the chosen Flow Collector. You can then choose to use data from all Flow Sources, or select Flow Sources one by one.

Finally, the 'Options' drawer allows you to define the 'Duration:', i.e. the time period the data is gathered over. Options include 'Last 1 Min', 'Last 2 Mins', 'Last 5 Mins' and 'Last 10 Mins'

Top 10 Port Receivers

The 'Top 10 Port Receivers' Dashlet will display the 'Top 10 Ports' in terms of Bytes and packets downloaded/recieved (i.e. data is being downloaded to Port 123), on a collector basis:

For example, in the above Dashlet we have chosen a collector, 'My Collector', which contains just the one Flow Source. The Dashlet therefore is showing that the port '2048' is responsible for 88.4% of all the bytes received/downloaded via the Flow Source (A Cisco router, in this example).

The configuration options for this Dashlet can be seen below:

You must choose a Flow Collector first, i.e. an Opsview Monitor master or slave server. After choosing the Flow Collector, the Flow Sources list will populate with all the Flow Sources created on the chosen Flow Collector. You can then choose to use data from all Flow Sources, or select Flow Sources one by one.

Finally, the 'Options' drawer allows you to define the 'Duration:', i.e. the time period the data is gathered over. Options include 'Last 1 Min', 'Last 2 Mins', 'Last 5 Mins' and 'Last 10 Mins'

Top 10 Transfers

The 'Top 10 Transfers' Dashlet will display the Top 10 transfers on a collector basis, including the transmitter (Host + port) and Reciever (Host + Port), along with the Bytes, Packets, and the % of the total Bytes transferred (allowing you to see if a single transfer is eating all of your bandwidth):

For example, in the above Dashlet we have chosen a collector, 'My Collector', which contains just the one Flow Source. The Dashlet therefore is showing that the port '2048' on the Host 'Cisco2611' is is responsible for 91.43% of all the bytes received/downloaded via the Flow Source (A Cisco router, in this example), and that the Host 'inst-ubuntu14-64.opsera.com' is responsible for transmitting the packets.

To view the transmitter port (as it is cut off in the Dashlet below), simply mouse over on the pie chart segment:

As above, we can now see that the transmitter port is 'ICMP'.

The configuration options for this Dashlet can be seen below:

You must choose a Flow Collector first, i.e. an Opsview Monitor master or slave server. After choosing the Flow Collector, the Flow Sources list will populate with all the Flow Sources created on the chosen Flow Collector. You can then choose to use data from all Flow Sources, or select Flow Sources one by one.

Finally, the 'Options' drawer allows you to define the 'Duration:', i.e. the time period the data is gathered over. Options include 'Last 1 Min', 'Last 2 Mins', 'Last 5 Mins' and 'Last 10 Mins'.

Network Analyzer

Learn how to use Network Analyzer