Checking Opsview Monitor Processes and Daemons

An overview on how to check processes and daemons in Opsview Monitor

Overview

We use Monit to monitor Opsview Monitor processes and give you the ability to start and stop them.

For more general information on Monit go to: https://mmonit.com/monit/

We refer to our integration of Monit as Opsview Watchdog.
You may check the current version of Monit being used by using the -V option

As root:
/opt/opsview/watchdog/bin/opsview-monit -V

How to use Opsview Watchdog

Watchdog contains a daemon, called opsview-monit, which must be started as the root user. This should be started automatically by the system.

However, the opsview user can get details from the daemon using the following script:

/opt/opsview/coreutils/bin/opsview_watchdog

This command supports three arguments with various actions. First you can view the summary of each service handled by the watchdog by running:

$ opsview_watchdog summary

The script can start, stop and reload individual Opsview Monitor processes:

$ opsview_watchdog <Process name> {start | stop | monitor | unmonitor | restart}

The script can also act on all Opsview Monitor processes:

$ opsview_watchdog all { start | stop | monitor | unmonitor | restart }

Advanced

If more debugging is required, you can query Watchdog as the root user. This script is located at /opt/opsview/watchdog/bin/opsview-monit and offers the same functionality as above, with additional options being available:

opsview-monit [options] {arguments}

Note: opsview-monit cannot be run as by the opsview user - use opsview_watchdog instead.

You can see process that are running and information about them by using either of the following commands:

# /opt/opsview/watchdog/bin/opsview-monit summary  { status | start | stop | restart | monitor | unmonitor }

The "status" option will give you a full list of information about each process, and "summary" will only show the current status of each process, much like opsview_watchdog.

The opsview-monit commands will also take process names as arguments if you need to check a single process. For example,​ if you wanted to see the status of opsview-web you can run:

# /opt/opsview/watchdog/bin/opsview-monit status opsview-web

Note: The order of the options is swapped compared to using opsview-watchdog!

When using opsview-monit some of the more advanced options look like this:

# /opt/opsview/watchdog/bin/opsview-monit {reload | quit | validate | procmatch <pattern> | reload} {all | <Process name> }

To view more usages of opsview-monit you can view the help document by running:

# /opt/opsview/watchdog/bin/opsview-monit -h

Web Interface

You can also view the status of processes in the Opsview Monitor web interface. By navigating to "CONFIGURATION > MY SYSTEM" you can see similar information to the opsview-monit status command under the System Overview tab.

This page will show you all the service names, the process names, which hosts they reside on, as well as their status. This page will also provide more information such as uptime, CPU usage, memory usage and more.

Troubleshooting

As an example, let's say that we have navigated to the web interface, but we are getting a 503 error. By using opsview_watchdog we can see if there are any processes that aren't running by using this command:

opsview@opsviewma1:~$ opsview_watchdog summary
+----------------------------------------+---------------+-------------------+
| Service                                | Status        | Monitoring Status |
+----------------------------------------+---------------+-------------------+
| Process 'opsview-watchdog'             | Running       | Monitored         |
| Process 'opsview-web'                  | Not monitored | Not monitored     |
| Process 'opsview-timeseriesrrdupdates' | Running       | Monitored         |
| Process 'opsview-timeseriesrrdqueries' | Running       | Monitored         |
| Process 'opsview-timeseriesenqueuer'   | Running       | Monitored         |
| Process 'opsview-timeseries'           | Running       | Monitored         |
| Process 'opsview-statechanges'         | Running       | Monitored         |
................................. [snipped] ................................. 
| System 'opsviewma1.opsview.com'        | Running       | Monitored         |
+----------------------------------------+---------------+-------------------+

Note: The above output has been shortened for clarity

Looking at the output we see that the opsview-web process isn't running. Let's start the processes using this command:

opsview@opsviewma1:~$ opsview_watchdog opsview-web start

We can then see that the process is being Initialized since the status will change to 'Not monitored - start pending'.

After a few seconds, we re-run opsview_watchdog summary and will see opsview-web is back to 'Monitored' and we can now access the web interface.

This process can be used to fix issues with the other processes on your system by using either the start or restart command. Keep in mind that all processes that you have licenses for should be marked as 'Running' or 'Accessible'.