## Overview

Easily and securely export high volumes of event data in real time to Splunk and other SIEM/analytics platforms

Highly-scalable Opsview Monitor is a solution for monitoring, aggregating, visualizing, alerting on, and drilling into event data from across arbitrarily-large, complex, multi-location, premise- and cloud-based enterprise IT estates. Enough so that it can be the "single pane of glass" that IT operations needs to work efficiently.

Opsview Monitor can also work as a take-off point -- providing data to other analytics, Security Information and Event Management (SIEM), bulk storage and other systems. The Opsview Monitor Results Exporter provides a complete, easy-to-use toolkit for extracting, filtering, and reformatting raw data directly from Opsview Monitor's message queue, and forwarding it to Splunk (SaaS or local) analytics, Enterprise Security, or to a host of other SIEM platform via syslog or HTTP.

## Getting Started

To use the Results Exporter, you first need to [Install the Results Exporter Component](🔗) and then configure an output.

Currently supported outputs and information on how to configure them:

  • [Syslog Outputs](🔗) - send information to a local or remote syslog server

  • [File Outputs](🔗) - send information to a file on the local filesystem

  • [HTTP Outputs](🔗) - send information over HTTP to a remote server - includes predefined supported types:

    • [Splunk Event Collector](🔗) - over unverified SSL.

    • [Certified Splunk Event Collector](🔗) - over verified SSL.

## Syslog Output

Results can be exported to a local or remote syslog server by configuring a syslog output, under the outputs section of your configuration file `/opt/opsview/resultsexporter/etc/resultsexporter.yaml`, e.g:



### Configuring a Syslog Output

The following options may be specified for your syslog output:

#### Required

None - if declared with no options, will log to `/dev/log` with all default settings.

#### Options

Parameter NameTypeDescriptionExampleDefault
hoststrThe hostname of the syslog server. If specified, _port_ must also be supplied.`host: '192.168.1.1'`Logs locally to `/dev/log`
portintThe port of the syslog server. If specified, _host_ must also be supplied.`port: 514`Logs locally to `/dev/log`
protocolstrThe transport protocol to use if using remote logging (not local `/dev/log`), either **udp** or **tcp**. It is recommended to use **udp**.`protocol: tcp``protocol: udp`
log_facilitystrThe facility used for syslog messages. Supported logging facilities: **auth**, **authpriv**, **cron**, **lpr**, **mail**, **daemon**, **ftp**, **kern**, **news**, **syslog**, **user**, **uucp**, **local0 - local7**`log_facility: local7``log_facility: user`
log_levelstrThe log level used for syslog messages. Supported logging levels (Highest priority to lowest): **critical**, **error**, **warning**, **notice**, **info**, **debug**`log_level: error``log_level: info`
log_date_formatstrThe format of the date in syslog messages. Can use any options listed in the [Log Date Format Strings](🔗) table below.`log_date_format: '(%Y) %m %d'``log_date_format: '%Y-%m-%d %H:%M:%S'`
log_formatstrThe format of syslog messages. Can use any options listed in the [Log Format Strings](🔗) table below - the `%(asctime)s` format option will match the format declared in _log_date_format_, if it has been specified.`log_format: 'msg: %(message)s'``log_format: '[opsview_resultsexporter %(asctime)s] %(message)s'`
filter
See the [Filtering](🔗) section for more details.

fields
See the [Field Mapping](🔗) section for more details.


#### Log Date Format Strings

DirectiveMeaning
`%a`Locale’s abbreviated weekday name.
`%A`Locale’s full weekday name.
`%b`Locale’s abbreviated month name.
`%B`Locale’s full month name.
`%c`Locale’s appropriate date and time representation.
`%d`Day of the month as a decimal number [01,31].
`%H`Hour (24-hour clock) as a decimal number [00,23].
`%I`Hour (12-hour clock) as a decimal number [01,12].
`%j`Day of the year as a decimal number [001,366].
`%m`Month as a decimal number [01,12].
`%M`Minute as a decimal number [00,59].
`%p`Locale’s equivalent of either AM or PM.
`%S`Second as a decimal number [00,61].
`%U`Week number of the year (Sunday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0.
`%w`Weekday as a decimal number [0(Sunday),6].
`%W`Week number of the year (Monday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Monday are considered to be in week 0.
`%x`Locale’s appropriate date representation.
`%X`Locale’s appropriate time representation.
`%y`Year without century as a decimal number [00,99].
`%Y`Year with century as a decimal number.
`%Z`Time zone name (no characters if no time zone exists).
`%%`A literal '%' character.

#### Log Format Strings

DirectiveMeaning
`%(message)s`The logged message.
`%(name)s`Name of the logger used to log the call.
`%(levelname)s`Text logging level for the message ('DEBUG', 'INFO', 'NOTICE', 'WARNING', 'ERROR', 'CRITICAL').
`%(asctime)s`Time when the log record was created.

## File Outputs

Results can be exported to a file on the local system by configuring a file output, under the outputs section of your configuration file `/opt/opsview/resultsexporter/etc/resultsexporter.yaml`, e.g:



### Configuring a File Output

The following options can be specified for your file output:

#### Required

Parameter NameTypeDescriptionExamples
pathstrThe path to the local file where this output will log messages. Note: The component will run as the opsview user, so the opsview home directory will be substituted for `~`.`path: '/var/log/resultsexporter.log'` <br><br> `path: '~/logs/my_file'`

#### Optional

Parameter NameTypeDescriptionExampleDefault
format_typestrThe format type of the messages logged to the file - see the [Formatting Messages](🔗) section for more details.`format_type: json``format_type: kvp`
filter
See the [Filtering](🔗) section for more details.

fields
See the [Field Mapping](🔗) section for more details.

message_format
The format of the messages logged to the file - see the [Formatting Messages](🔗) section for more details.


## HTTP Outputs

Results can be exported via HTTP to an external service by configuring a HTTP output, under the outputs section of your configuration file `/opt/opsview/resultsexporter/etc/resultsexporter.yaml`, e.g:



### Configuring a Custom HTTP Output

#### Required

Parameter NameTypeDescriptionExample(s)
endpointstrThe endpoint where this output will send requests. By default, if no port is specified in the endpoint string, the component will attempt to connect to port 80. If the scheme in the endpoint string is not https, the component will default to http.`endpoint: 'http://www.mywebsite.com:8000/resultscollector'`

#### Optional

Parameter NameTypeDescriptionExample(s)Default
headersdictThe headers to be included in the HTTP request.`headers: Authorization: 'Basic YWxhZGRpbjpvcGVuc2VzYW1l' Content-Type: 'text/html; charset=utf-8'``headers: {}`
typestrThe HTTP output type. If not custom then refer to the table below instead for options.
`type: custom`
bodystrThe format of the request body. The `%(data)s` format string will be replaced by the data being sent in each post request (your messages after message formatting). **NOTE:** For JSON format, the messages will be concatenated into a JSON array before being substituted into your specified body format.`body: '{"my_data": %(data)s}'` `body: 'data_prefix %(data)s data_suffix'``body: '%(data)s'`
ssl_optionsdictThe ssl options to be used. Currently supported options: **insecure** (bool), **cert_reqs** (str), **ssl_version** (str), **ca_certs** (str), **ciphers** (str), **keyfile** (str), **certfile** (str)`ssl_options: insecure: False cert_reqs: CERT_REQUIRED ssl_version: PROTOCOL_TLS ca_certs: '/path/to/ca_certs' ciphers: 'HIGH+TLSv1.2:!MD5:!SHA1' keyfile: '/path/to/keyfile' certfile: '/path/to/certfile'``ssl_options: insecure: True cert_reqs: CERT_NONE ssl_version: PROTOCOL_TLS ca_certs: null ciphers: null keyfile: null certfile: null`
format_typestrThe format type of the messages logged to the file - see the [Formatting Messages](🔗) section below for more details.
`format_type: json`
filter
See the [Filtering](🔗) section below for more details.

fields
See the [Field Mapping](🔗) section below for more details.

message_format
The format of the messages logged to the file - see the [Formatting Messages](🔗) section below.


### Configuring a Predefined HTTP Output

The following options can be specified for an HTTP output with a predefined `type`: spunk, splunk-cert

#### splunk

Results can be exported via unverified HTTPS to Splunk by configuring a HTTP output with `type: splunk` under the http section of your configuration file outputs within `/opt/opsview/resultsexporter/etc/resultsexporter.yaml`, e.g:



The following parameters are required for an HTTP Splunk output:

Parameter NameTypeDescriptionExample(s)
hoststrThe hostname/IP Address of your Splunk Server where you have set up [Splunk HTTP Event Collection](🔗).`host: '192.168.1.1'`
portintThe port specified in the Global Settings of your Splunk HTTP Event Collectors.`port: 8088`
tokenstrThe token relating to your specific Splunk HTTP Event Collector.`token: '103a4f2f-023f-0cff-f9d7-3413d52b4b2b'`

#### splunk-cert

Results can be exported via HTTPS to Splunk (using a client certificate) by configuring a HTTP output with type splunk-cert under the http section of your configuration file outputs within `/opt/opsview/resultsexporter/etc/resultsexporter.yaml`, e.g:



The following parameters are required for an HTTP Splunk output:

Parameter NameTypeDescriptionExample(s)
hoststrThe hostname/IP Address of your Splunk Server where you have set up Splunk HTTP Event Collection.`host: '192.168.1.1'`
portintThe port specified in the Global Settings of your Splunk HTTP Event Collectors.`port: 8088`
tokenstrThe token relating to your specific Splunk HTTP Event Collector.`token: '103a4f2f-023f-0cff-f9d7-3413d52b4b2b'`
ca_certsstrThe path to your CA (Certificate Authority) Certificate(s).`ca_certs: '/mycerts/ca.crt'`
certfilestrThe path to your client certificate.`certfile: '/mycerts/client.crt'`
keyfilestrThe path to the private key for your client certificate.`keyfile: '/mycerts/client.key'`

Note: If your client certificate and key are both within the same `.pem` file then you can simply list that file path for both `certfile` and `keyfile`.

## Field Mapping

The Results Exporter allows you to transform the messages as they are exported, by specifying exactly which message fields should be present in the exported result, so you can remove details from the messages you do not want or need:

Within `/opt/opsview/resultsexporter/etc/resultsexporter.yaml`:


Parameter NameTypeDescriptionExampleDefault
fieldsdictThe field mapping to apply to your output.`fields: - hostname - stdout``fields: - hostname - servicecheckname - current_state - problem_has_been_acknowledged - is_hard_state - check_attempt - last_check - execution_time - stdout`

### Specifying Fields

The fields of messages being exported are fully configurable. To select the fields that should be included in exported messages, list the keys under the fields section of an output. For example to include the `hostname`, `servicecheckname`, `current_state` and `stdout` fields, add them to the fields section:


##### Simple Examples


#### Mapping and renaming fields

Users can also specify custom keys which are given values based on a mapping. For example to retrieve the `host_state` as a new field with name `host_state_string`, and value `UP` or `DOWN` instead of `0` or `1`:



In this example, the value of `host_state` determines behaviour as below:

Value of `host_state`Behaviour
'0'`host_state_string` will be added to result with value 'UP'
'1'`host_state_string` will be added to result with value 'DOWN'
Anything else`host_state_string` will not be added to result

A default value can also be specified. For example, in the example below, if the value of the `hostname` field does not match 'web-server' or 'email-server' , the value will be set to 'AllCompany'. If a default value is not specified and the key does not match any of the keys provided, the field will be omitted from the output.



This example results in behaviour as below:

Value of `hostname`Behaviour
'web-server'`department` will be added to result with value 'Engineering'
'email-server'`department` will be added to result with value 'BS'
Anything else`department` will be added to result with value 'AllCompany'

Fields can also be added where the value is always constant.



This example results in behaviour as below:

Value of `hostname`Behaviour
Anything`department` will be added to result with value 'AllCompany'

Mapped values can refer to any (original) message fields, by using the format syntax: `%(<field>)s`, as shown in the example below.



This example results in behaviour as below:

Value of `check_state` (service check name is "Server Connectivity")Behaviour
'0'`priority_msg` will be added to result with value 'Server Connectivity HAS PRIORITY: LOW (OK)'
'2'`priority_msg` will be added to result with value 'Server Connectivity HAS PRIORITY: HIGH (CRITICAL)'
Anything else (here called X)`priority_msg` will be added to result with value 'Server Connectivity HAS PRIORITY: MEDIUM (X)'

This allows message fields to be 'renamed' easily if required by providing a one to one mapping with the original message field. For example to 'rename' the `hostname` field to `name`:



This example results in behaviour as below:

Value of `hostname`Behaviour
Anything (here called X)`name` will be added to result with value 'X'



Note: if you change your mapping values then you should review all your filters to ensure they will still work as expected

##### Reusing fields

To avoid duplication, the fields can be defined once using the `&` (yaml anchor) operator and reused multiple times using the `*` (anchor reference) operator.



Alternatively, anchors can be declared as a list, and can have optional names for clarity, as in this example:



#### Performance Data Fields

The Results Exporter component exposes the individual metrics within the performance data of each message. To include the raw performance data string in your exported message, include the `perf_data_raw` field within your list of fields e.g for a service check with performance data of:



To include the entire performance data as a nested structure within your message, include the `perf_data` field:



To include some of the nested fields, but not all, you can specify specific named metrics as below:



### Supported fields

FieldTypeExampleDescription
check_attemptint`1`The current check attempt number (0 < check_attempt < max_check_attempts).
current_stateint`0`Current state of the check.
downtime_depthint`0`The number of active downtimes an object is currently included in (0 indicates not in downtime).
early_timeoutbool`false`Set if the execution of the plugin timed out.
end_timefloat`1543504202.051796`The epoch time when the plugin execution completed.
execution_timefloat`4.0345320702`How long the plugin took to execute.
host_idint`26`The Opsview Host ID for the host relating to this host/service check.
host_stateint`0`The state the host is currently known to be in.
hostnamestring`ldap-cache1.opsview.com`Name of the Opsview Monitor host that produced the result message.
init_timefloat`1543504198.002325`The time when the execution request message was created.
is_flappingbool`false`Has flapping been detected (repeated OK->non-OK results subsequent checks).
is_hard_host_statebool`true`Is the host in a HARD or SOFT state.
is_hard_statebool`true`Is the check in a HARD or SOFT state.
is_hard_state_changebool`true`Has this result just changed from SOFT to HARD.
is_passive_resultbool`true`Is this result from a passive or an active check.
is_state_changebool`false`Has a change of state been detected.
last_checkint`1543504198`Integer epoch time of when the check last ran.
last_hard_stateint`0`The value of the last HARD state.
last_hard_state_changeint`1543434256`Epoch value of when the check last changed to a HARD state.
last_state_changeint`1543486858`Epoch value of when the check last changed state (SOFT or HARD).
latencyfloat`0.0149388313`The difference between when the check was scheduled to run and when it actually ran.
max_check_attemptsint`3`Number of check attempts before a SOFT error is counted as HARD.
object_idint`953`The Opsview Object ID number for this host/service.
object_id_typestring`service`Whether this is a host or a service check.
perf_data_rawstring`rta=0.034ms;500.000;1000.000;0; pl=0%;80;100;; rtmax=0.113ms;;;; rtmin=0.011ms;;;;`The performance data returned from the host/service check.
perf_data
Adds the entire nested structure of perf_data metrics to the message (JSON/YAML/XML), or shorthand for adding each of the metrics as a string to the message (KVP). See Formatting Messages for examples.
perf_data.some-metric-name
Adds that metric to the nested structure of perf_data metrics and adds the nested structure to the message if not already present (JSON/YAML/XML), or add that individual metric as a string to the message (KVP).
prev_stateint`0`The state returned by the previous check.
problem_has_been_acknowledgedbool`false`Has a non-OK state been acknowledged.
servicechecknamestring`TCP/IP`Service Check name, or null for a host check.
start_timefloat`1543504198.017264`The time the plugin started executing.
stdoutstring`PING OK - Packet loss = 0%RTA = 14.85 ms`Output from the plugin.
timestampfloat`1543504202.057018`Time the message was created.
metadata
Adds the nested structure of metadata metrics to the message (JSON/YAML/XML), or shorthand for adding each of the metrics as a string to the message (KVP).This currently includes: **hostname_run_on** (the name of the host machine that the service check was run on).
ack_ref
Internal Use.
broadcast
Internal Use.
job_ref
Internal Use.
message_class
Internal Use.
message_source
Internal Use.
ref
Internal Use.

## Filtering

A powerful feature of the Results Exporter is the ability to select exactly which messages you want to be exported via each output, using a filter string. Any message that meets the condition specified in the filter will be processed and exported by that output e.g:


Parameter NameTypeDescriptionExample(s)Default
filterstrThe filter string to apply to your output.`filter: '(current_state !~ "OK")'``filter: ''`

### Specifying Filters

Using a filter allows you to focus on exporting the results that are important and ignore anything that isn't relevant to your chosen reporting/searching tool.

### Inbuilt Filters

OperatorDescription
(empty string)allows every message
*allows every message
!*allows no messages

A filter string can also consist of a comparison between a key within the message and a value (the value for the key within each message being filtered).

More complex filters can be written by combining filter strings using the `&&` (logical and) and `||` (logical or) operators.

#### Supported comparisons

OperatorDescription
==is equal to
!=is not equal to
>=is greater than or equal to
<=is less than or equal to
~contains
!~does not contain
<is less than
>is greater than
@matches (regex)
!@does not match (regex)

### Supported Fields

Within your filter, you can refer to any field listed as supported in Field Mapping, with the exception of `perf_data`, `perf_data_raw` and any extracted performance data fields. These are not supported by the filter.

##### Simple Filter Examples

##### Complex Filter Examples


NOTE: It is advised to surround your filters with single quotes:



If your filter is surrounded in double quotes, you will need to escape backlashes in your regular expressions:



#### Reusing Filters

To avoid duplication, the filters can be defined once using the `&` (yaml anchor) operator and reused multiple times using the `*` (anchor reference) operator.



Alternatively, anchors can be declared as a list, and can have optional names for clarity, as in this example:



#### Multi-line Filters

Multi-line filters are possible as long as the entire filter is quoted - this can add clarity for complex filters as seen below:



## Formatting Messages

The Results Exporter allows you to declare the format type of the exported messages for file and HTTP outputs, as well as adding any additional information to each message using a format string:


Parameter NameTypeDescriptionExample(s)Default
format_typestrSupported formats: kvp, json, yaml, xml`format_type: xml`See output options.
message_formatstrThe format of each message being exported. The %(message)s format string will be expanded into the exported message formatted in the markup language, as specified by the format_type field.`message_format: '<mytag />%(message)s'``message_format: '%(message)s'`

### Format Types

The list of currently supported format types is as follows:

  • XML

  • JSON

  • KVP (Key Value Pairs)

  • YAML

### Example Job Result Message:



#### Formatted into kvp:



#### Formatted into json:



#### Formatted into xml:



#### Formatted into yaml: