mirror of
https://github.com/drewkerrigan/nagios-http-json.git
synced 2026-02-12 01:51:01 +01:00
Compare commits
41 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
9c0c59d6c1 | ||
|
e2fce71d5a | ||
|
47547951cf | ||
|
|
7858382bbe | ||
|
|
1173420803 | ||
|
|
bcc36a6e95 | ||
|
|
d98d0396b2 | ||
|
|
8437c464e5 | ||
|
|
df2bbdbf51 | ||
|
|
823fc275c9 | ||
|
|
18b0898e72 | ||
|
|
95318954bf | ||
|
|
8e469e3d98 | ||
|
|
29f8d892ee | ||
|
cbdb884dc7 | ||
|
|
3a108aef5e | ||
|
81522fa9ab | ||
|
|
27eaaf0842 | ||
|
9dd6323b85 | ||
|
67136a4a2b | ||
|
|
d164a1250c | ||
|
|
89f42c15a0 | ||
|
|
1e707a4b6a | ||
|
|
9656265439 | ||
|
|
e463369671 | ||
|
357c2240ba | ||
|
42d1e08037 | ||
|
4950225393 | ||
|
|
9be6a709a2 | ||
|
|
06fab10fe2 | ||
|
|
7bdc802c2d | ||
|
|
ed7bc7175b | ||
|
|
4180ec2066 | ||
|
a0d0773d1a | ||
|
fbebf05f76 | ||
|
6f9048fc75 | ||
|
|
5bb09cd362 | ||
|
|
568fa6e4d0 | ||
|
|
f63ac180b6 | ||
|
|
070047cf55 | ||
|
|
8adcf2ff07 |
413
README.md
413
README.md
@@ -2,14 +2,198 @@
|
||||
|
||||
This is a generic plugin for Nagios which checks json values from a given HTTP endpoint against argument specified rules and determines the status and performance data for that service.
|
||||
|
||||
### Installation
|
||||
## Links
|
||||
|
||||
#### Requirements
|
||||
* [CLI Usage](#cli-usage)
|
||||
* [Examples](#examples)
|
||||
* [Riak Stats](docs/RIAK.md)
|
||||
* [Docker](docs/DOCKER.md)
|
||||
* [Nagios Installation](#nagios-installation)
|
||||
|
||||
* Nagios
|
||||
* Python
|
||||
## CLI Usage
|
||||
|
||||
### Nagios Configuration
|
||||
Executing `./check_http_json.py -h` will yield the following details:
|
||||
|
||||
```
|
||||
usage: check_http_json.py [-h] [-d] [-s] -H HOST [-k] [-V] [--cacert CACERT]
|
||||
[--cert CERT] [--key KEY] [-P PORT] [-p PATH]
|
||||
[-t TIMEOUT] [-B AUTH] [-D DATA] [-A HEADERS]
|
||||
[-f SEPARATOR]
|
||||
[-w [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]]]
|
||||
[-c [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]]]
|
||||
[-e [KEY_LIST [KEY_LIST ...]]]
|
||||
[-E [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]]]
|
||||
[-q [KEY_VALUE_LIST [KEY_VALUE_LIST ...]]]
|
||||
[-Q [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]]]
|
||||
[-u [KEY_VALUE_LIST_UNKNOWN [KEY_VALUE_LIST_UNKNOWN ...]]]
|
||||
[-y [KEY_VALUE_LIST_NOT [KEY_VALUE_LIST_NOT ...]]]
|
||||
[-Y [KEY_VALUE_LIST_NOT_CRITICAL [KEY_VALUE_LIST_NOT_CRITICAL ...]]]
|
||||
[-m [METRIC_LIST [METRIC_LIST ...]]]
|
||||
|
||||
Check HTTP JSON Nagios Plugin
|
||||
|
||||
Generic Nagios plugin which checks json values from a given endpoint against
|
||||
argument specified rules and determines the status and performance data for
|
||||
that service.
|
||||
|
||||
Version: 1.4.0 (2019-05-09)
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-d, --debug debug mode
|
||||
-s, --ssl use TLS to connect to remote host
|
||||
-H HOST, --host HOST remote host to query
|
||||
-k, --insecure do not check server SSL certificate
|
||||
-V, --version print version of this plugin
|
||||
--cacert CACERT SSL CA certificate
|
||||
--cert CERT SSL client certificate
|
||||
--key KEY SSL client key ( if not bundled into the cert )
|
||||
-P PORT, --port PORT TCP port
|
||||
-p PATH, --path PATH Path
|
||||
-t TIMEOUT, --timeout TIMEOUT
|
||||
Connection timeout (seconds)
|
||||
-B AUTH, --basic-auth AUTH
|
||||
Basic auth string "username:password"
|
||||
-D DATA, --data DATA The http payload to send as a POST
|
||||
-A HEADERS, --headers HEADERS
|
||||
The http headers in JSON format.
|
||||
-f SEPARATOR, --field_separator SEPARATOR
|
||||
JSON Field separator, defaults to "."; Select element
|
||||
in an array with "(" ")"
|
||||
-w [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]], --warning [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]]
|
||||
Warning threshold for these values
|
||||
(key1[>alias],WarnRange key2[>alias],WarnRange).
|
||||
WarnRange is in the format [@]start:end, more
|
||||
information at nagios-plugins.org/doc/guidelines.html.
|
||||
-c [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]], --critical [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]]
|
||||
Critical threshold for these values
|
||||
(key1[>alias],CriticalRange
|
||||
key2[>alias],CriticalRange. CriticalRange is in the
|
||||
format [@]start:end, more information at nagios-
|
||||
plugins.org/doc/guidelines.html.
|
||||
-e [KEY_LIST [KEY_LIST ...]], --key_exists [KEY_LIST [KEY_LIST ...]]
|
||||
Checks existence of these keys to determine status.
|
||||
Return warning if key is not present.
|
||||
-E [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]], --key_exists_critical [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]]
|
||||
Same as -e but return critical if key is not present.
|
||||
-q [KEY_VALUE_LIST [KEY_VALUE_LIST ...]], --key_equals [KEY_VALUE_LIST [KEY_VALUE_LIST ...]]
|
||||
Checks equality of these keys and values
|
||||
(key[>alias],value key2,value2) to determine status.
|
||||
Multiple key values can be delimited with colon
|
||||
(key,value1:value2). Return warning if equality check
|
||||
fails
|
||||
-Q [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]], --key_equals_critical [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]]
|
||||
Same as -q but return critical if equality check
|
||||
fails.
|
||||
-u [KEY_VALUE_LIST_UNKNOWN [KEY_VALUE_LIST_UNKNOWN ...]], --key_equals_unknown [KEY_VALUE_LIST_UNKNOWN [KEY_VALUE_LIST_UNKNOWN ...]]
|
||||
Same as -q but return unknown if equality check fails.
|
||||
-y [KEY_VALUE_LIST_NOT [KEY_VALUE_LIST_NOT ...]], --key_not_equals [KEY_VALUE_LIST_NOT [KEY_VALUE_LIST_NOT ...]]
|
||||
Checks equality of these keys and values
|
||||
(key[>alias],value key2,value2) to determine status.
|
||||
Multiple key values can be delimited with colon
|
||||
(key,value1:value2). Return warning if equality check
|
||||
succeeds
|
||||
-Y [KEY_VALUE_LIST_NOT_CRITICAL [KEY_VALUE_LIST_NOT_CRITICAL ...]], --key_not_equals_critical [KEY_VALUE_LIST_NOT_CRITICAL [KEY_VALUE_LIST_NOT_CRITICAL ...]]
|
||||
Same as -q but return critical if equality check
|
||||
succeeds.
|
||||
-m [METRIC_LIST [METRIC_LIST ...]], --key_metric [METRIC_LIST [METRIC_LIST ...]]
|
||||
Gathers the values of these keys (key[>alias],
|
||||
UnitOfMeasure,WarnRange,CriticalRange,Min,Max) for
|
||||
Nagios performance data. More information about Range
|
||||
format and units of measure for nagios can be found at
|
||||
nagios-plugins.org/doc/guidelines.html Additional
|
||||
formats for this parameter are: (key[>alias]),
|
||||
(key[>alias],UnitOfMeasure),
|
||||
(key[>alias],UnitOfMeasure,WarnRange, CriticalRange).
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Key Naming
|
||||
|
||||
**Data for key** `value`:
|
||||
|
||||
{ "value": 1000 }
|
||||
|
||||
**Data for key** `capacity.value`:
|
||||
|
||||
{
|
||||
"capacity": {
|
||||
"value": 1000
|
||||
}
|
||||
}
|
||||
|
||||
**Data for key** `(0).capacity.value`:
|
||||
|
||||
[
|
||||
{
|
||||
"capacity": {
|
||||
"value": 1000
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
**Data for separator** `-f _` **and key** `(0)_gauges_jvm.buffers.direct.capacity_value`:
|
||||
|
||||
[
|
||||
{
|
||||
"gauges": {
|
||||
"jvm.buffers.direct.capacity":
|
||||
"value": 1000
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
**Data for keys** `ring_members(0)`, `ring_members(1)`, `ring_members(2)`:
|
||||
|
||||
{
|
||||
"ring_members": [
|
||||
"riak1@127.0.0.1",
|
||||
"riak2@127.0.0.1",
|
||||
"riak3@127.0.0.1"
|
||||
]
|
||||
}
|
||||
|
||||
### Thresholds and Ranges
|
||||
|
||||
**Data**:
|
||||
|
||||
{ "metric": 1000 }
|
||||
|
||||
#### Relevant Commands
|
||||
|
||||
* **Warning:** `./check_http_json.py -H <host>:<port> -p <path> -w "metric,RANGE"`
|
||||
* **Critical:** `./check_http_json.py -H <host>:<port> -p <path> -c "metric,RANGE"`
|
||||
* **Metrics with Warning:** `./check_http_json.py -H <host>:<port> -p <path> -w "metric,RANGE"`
|
||||
* **Metrics with Critical:**
|
||||
|
||||
./check_http_json.py -H <host>:<port> -p <path> -w "metric,,,RANGE"
|
||||
./check_http_json.py -H <host>:<port> -p <path> -w "metric,,,,MIN,MAX"
|
||||
|
||||
#### Range Definitions
|
||||
|
||||
* **Format:** [@]START:END
|
||||
* **Generates a Warning or Critical if...**
|
||||
* **Value is less than 0 or greater than 1000:** `1000` or `0:1000`
|
||||
* **Value is greater than or equal to 1000, or less than or equal to 0:** `@1000` or `@0:1000`
|
||||
* **Value is less than 1000:** `1000:`
|
||||
* **Value is greater than 1000:** `~:1000`
|
||||
* **Value is greater than or equal to 1000:** `@1000:`
|
||||
|
||||
More info about Nagios Range format and Units of Measure can be found at [https://nagios-plugins.org/doc/guidelines.html](https://nagios-plugins.org/doc/guidelines.html).
|
||||
|
||||
#### Using Headers
|
||||
|
||||
* `./check_http_json.py -H <host>:<port> -p <path> -A '{"content-type": "application/json"}' -w "metric,RANGE"`
|
||||
|
||||
## Nagios Installation
|
||||
|
||||
### Requirements
|
||||
|
||||
* Python 2.7
|
||||
|
||||
### Configuration
|
||||
|
||||
Assuming a standard installation of Nagios, the plugin can be executed from the machine that Nagios is running on.
|
||||
|
||||
@@ -44,225 +228,6 @@ define command{
|
||||
|
||||
More info about options in Usage.
|
||||
|
||||
### CLI Usage
|
||||
|
||||
Executing `./check_http_json.py -h` will yield the following details:
|
||||
|
||||
```
|
||||
usage: check_http_json.py [-h] [-d] [-s] -H HOST [-P PORT] [-p PATH]
|
||||
[-t TIMEOUT] [-B AUTH] [-D DATA] [-f SEPARATOR]
|
||||
[-w [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]]]
|
||||
[-c [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]]]
|
||||
[-e [KEY_LIST [KEY_LIST ...]]]
|
||||
[-E [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]]]
|
||||
[-q [KEY_VALUE_LIST [KEY_VALUE_LIST ...]]]
|
||||
[-Q [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]]]
|
||||
[-m [METRIC_LIST [METRIC_LIST ...]]]
|
||||
|
||||
Nagios plugin which checks json values from a given endpoint against argument
|
||||
specified rules and determines the status and performance data for that
|
||||
service
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-d, --debug Debug mode.
|
||||
-s, --ssl HTTPS mode.
|
||||
-H HOST, --host HOST Host.
|
||||
-P PORT, --port PORT TCP port
|
||||
-p PATH, --path PATH Path.
|
||||
-t TIMEOUT, --timeout TIMEOUT
|
||||
Connection timeout (seconds)
|
||||
-B AUTH, --basic-auth AUTH
|
||||
Basic auth string "username:password"
|
||||
-D DATA, --data DATA The http payload to send as a POST
|
||||
-f SEPARATOR, --field_separator SEPARATOR
|
||||
Json Field separator, defaults to "." ; Select element
|
||||
in an array with "(" ")"
|
||||
-w [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]], --warning [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]]
|
||||
Warning threshold for these values
|
||||
(key1[>alias],WarnRange key2[>alias],WarnRange).
|
||||
WarnRange is in the format [@]start:end, more
|
||||
information at nagios-plugins.org/doc/guidelines.html.
|
||||
-c [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]], --critical [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]]
|
||||
Critical threshold for these values
|
||||
(key1[>alias],CriticalRange
|
||||
key2[>alias],CriticalRange. CriticalRange is in the
|
||||
format [@]start:end, more information at nagios-
|
||||
plugins.org/doc/guidelines.html.
|
||||
-e [KEY_LIST [KEY_LIST ...]], --key_exists [KEY_LIST [KEY_LIST ...]]
|
||||
Checks existence of these keys to determine status.
|
||||
Return warning if key is not present.
|
||||
-E [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]], --key_exists_critical [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]]
|
||||
Same as -e but return critical if key is not present.
|
||||
-q [KEY_VALUE_LIST [KEY_VALUE_LIST ...]], --key_equals [KEY_VALUE_LIST [KEY_VALUE_LIST ...]]
|
||||
Checks equality of these keys and values
|
||||
(key[>alias],value key2,value2) to determine status.
|
||||
Multiple key values can be delimited with colon
|
||||
(key,value1:value2). Return warning if equality check
|
||||
fails
|
||||
-Q [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]], --key_equals_critical [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]]
|
||||
Same as -q but return critical if equality check
|
||||
fails.
|
||||
-m [METRIC_LIST [METRIC_LIST ...]], --key_metric [METRIC_LIST [METRIC_LIST ...]]
|
||||
Gathers the values of these keys (key[>alias],UnitOfMe
|
||||
asure,WarnRange,CriticalRange,Min,Max) for Nagios
|
||||
performance data. More information about Range format
|
||||
and units of measure for nagios can be found at
|
||||
nagios-plugins.org/doc/guidelines.html Additional
|
||||
formats for this parameter are: (key[>alias]),
|
||||
(key[>alias],UnitOfMeasure),
|
||||
(key[>alias],UnitOfMeasure,WarnRange,CriticalRange).
|
||||
```
|
||||
|
||||
Access a specific JSON field by following this syntax: `alpha.beta.gamma(3).theta.omega(0)`
|
||||
Dots are field separators (changeable), parentheses are for entering arrays.
|
||||
|
||||
If the root of the JSON data is itself an array like the following:
|
||||
|
||||
```
|
||||
[
|
||||
{ "gauges": { "jvm.buffers.direct.capacity": {"value": 215415}}}
|
||||
]
|
||||
```
|
||||
|
||||
The beginning of the key should start with ($index) as in this example:
|
||||
|
||||
```
|
||||
./check_http_json.py -H localhost:8081 -p metrics --key_exists "(0)_gauges_jvm.buffers.direct.capacity_value" -f _
|
||||
```
|
||||
|
||||
More info about Nagios Range format and Units of Measure can be found at [https://nagios-plugins.org/doc/guidelines.html](https://nagios-plugins.org/doc/guidelines.html).
|
||||
|
||||
### Docker Info Example Plugin
|
||||
|
||||
#### Description
|
||||
|
||||
Let's say we want to use `check_http_json.py` to read from Docker's `/info` HTTP API endpoint with the following parameters:
|
||||
|
||||
##### Connection information
|
||||
|
||||
* Host = 127.0.0.1:4243
|
||||
* Path = /info
|
||||
|
||||
##### Rules for "aliveness"
|
||||
|
||||
* Verify that the key `Containers` exists in the outputted JSON
|
||||
* Verify that the key `IPv4Forwarding` has a value of `1`
|
||||
* Verify that the key `Debug` has a value less than or equal to `2`
|
||||
* Verify that the key `Images` has a value greater than or equal to `1`
|
||||
* If any of these criteria are not met, report a WARNING to Nagios
|
||||
|
||||
##### Gather Metrics
|
||||
|
||||
* Report value of the key `Containers` with a MinValue of 0 and a MaxValue of 1000 as performance data
|
||||
* Report value of the key `Images` as performance data
|
||||
* Report value of the key `NEventsListener` as performance data
|
||||
* Report value of the key `NFd` as performance data
|
||||
* Report value of the key `NGoroutines` as performance data
|
||||
* Report value of the key `SwapLimit` as performance data
|
||||
|
||||
#### Service Definition
|
||||
|
||||
`localhost.cfg`
|
||||
|
||||
```
|
||||
define service {
|
||||
use local-service
|
||||
host_name localhost
|
||||
service_description Docker info status checker
|
||||
check_command check_docker
|
||||
}
|
||||
```
|
||||
|
||||
#### Command Definition with Arguments
|
||||
|
||||
`commands.cfg`
|
||||
|
||||
```
|
||||
define command{
|
||||
command_name check_docker
|
||||
command_line /usr/bin/python /usr/local/nagios/libexec/plugins/check_http_json.py -H 127.0.0.1:4243 -p info -e Containers -q IPv4Forwarding,1 -w Debug,2:2 -c Images,1:1 -m Containers,0:250,0:500,0,1000 Images NEventsListener NFd NGoroutines SwapLimit
|
||||
}
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
OK: Status OK.|'Containers'=1;0;1000 'Images'=11;0;0 'NEventsListener'=3;0;0 'NFd'=10;0;0 'NGoroutines'=14;0;0 'SwapLimit'=1;0;0
|
||||
```
|
||||
|
||||
### Docker Container Monitor Example Plugin
|
||||
|
||||
`check_http_json.py` is generic enough to read and evaluate rules on any HTTP endpoint that returns JSON. In this example we'll get the status of a specific container using it's ID which camn be found by using the list containers endpoint (`curl http://127.0.0.1:4243/containers/json?all=1`).
|
||||
|
||||
##### Connection information
|
||||
|
||||
* Host = 127.0.0.1:4243
|
||||
* Path = /containers/2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615/json
|
||||
|
||||
##### Rules for "aliveness"
|
||||
|
||||
* Verify that the key `ID` exists and is equal to the value `2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615`
|
||||
* Verify that the key `State.Running` has a value of `True`
|
||||
|
||||
#### Service Definition
|
||||
|
||||
`localhost.cfg`
|
||||
|
||||
```
|
||||
define service {
|
||||
use local-service
|
||||
host_name localhost
|
||||
service_description Docker container liveness check
|
||||
check_command check_my_container
|
||||
}
|
||||
```
|
||||
|
||||
#### Command Definition with Arguments
|
||||
|
||||
`commands.cfg`
|
||||
|
||||
```
|
||||
define command{
|
||||
command_name check_my_container
|
||||
command_line /usr/bin/python /usr/local/nagios/libexec/plugins/check_http_json.py -H 127.0.0.1:4243 -p /containers/2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615/json -q ID,2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615 State.Running,True
|
||||
}
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
WARNING: Status check failed, reason: Value True for key State.Running did not match.
|
||||
```
|
||||
|
||||
The plugin threw a warning because the Container ID I used on my system has the following State object:
|
||||
|
||||
```
|
||||
u'State': {...
|
||||
u'Running': False,
|
||||
...
|
||||
```
|
||||
|
||||
If I change the command to have the parameter -q parameter `State.Running,False`, the output becomes:
|
||||
|
||||
```
|
||||
OK: Status OK.
|
||||
```
|
||||
|
||||
### Dropwizard / Fieldnames Containing '.' Example
|
||||
|
||||
Simply choose a separator to deal with data such as this:
|
||||
|
||||
```
|
||||
{ "gauges": { "jvm.buffers.direct.capacity": {"value": 215415}}}
|
||||
```
|
||||
|
||||
In this example I've chosen `_` to separate `guages` from `jvm` and `capacity` from `value`. The CLI invocation then becomes:
|
||||
|
||||
```
|
||||
./check_http_json.py -H localhost:8081 -p metrics --key_exists gauges_jvm.buffers.direct.capacity_value -f _
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2014-2015 Drew Kerrigan.
|
||||
|
||||
@@ -1,39 +1,48 @@
|
||||
#!/usr/bin/python
|
||||
#!/usr/bin/python2.7
|
||||
|
||||
plugin_description = \
|
||||
"""
|
||||
Check HTTP JSON Nagios Plugin
|
||||
|
||||
Generic Nagios plugin which checks json values from a given endpoint against argument specified rules
|
||||
and determines the status and performance data for that service.
|
||||
Generic Nagios plugin which checks json values from a given endpoint against
|
||||
argument specified rules and determines the status and performance data for
|
||||
that service.
|
||||
"""
|
||||
|
||||
import httplib, urllib, urllib2, base64
|
||||
import urllib2
|
||||
import base64
|
||||
import json
|
||||
import argparse
|
||||
import sys
|
||||
import ssl
|
||||
from pprint import pprint
|
||||
from urllib2 import HTTPError
|
||||
from urllib2 import URLError
|
||||
|
||||
# TEST = False
|
||||
|
||||
|
||||
OK_CODE = 0
|
||||
WARNING_CODE = 1
|
||||
CRITICAL_CODE = 2
|
||||
UNKNOWN_CODE = 3
|
||||
|
||||
__version__ = '1.4.0'
|
||||
__version_date__ = '2019-05-09'
|
||||
|
||||
class NagiosHelper:
|
||||
"""Help with Nagios specific status string formatting."""
|
||||
message_prefixes = {OK_CODE: 'OK', WARNING_CODE: 'WARNING', CRITICAL_CODE: 'CRITICAL', UNKNOWN_CODE: 'UNKNOWN'}
|
||||
message_prefixes = {OK_CODE: 'OK',
|
||||
WARNING_CODE: 'WARNING',
|
||||
CRITICAL_CODE: 'CRITICAL',
|
||||
UNKNOWN_CODE: 'UNKNOWN'}
|
||||
performance_data = ''
|
||||
warning_message = ''
|
||||
critical_message = ''
|
||||
unknown_message = ''
|
||||
|
||||
def getMessage(self):
|
||||
"""Build a status-prefixed message with optional performance data generated externally"""
|
||||
text = "%s: Status %s." % (self.message_prefixes[self.getCode()], self.message_prefixes[self.getCode()])
|
||||
"""Build a status-prefixed message with optional performance data
|
||||
generated externally"""
|
||||
text = "%s: Status %s." % (self.message_prefixes[self.getCode()],
|
||||
self.message_prefixes[self.getCode()])
|
||||
text += self.warning_message
|
||||
text += self.critical_message
|
||||
text += self.unknown_message
|
||||
@@ -53,17 +62,23 @@ class NagiosHelper:
|
||||
|
||||
def append_warning(self, warning_message):
|
||||
self.warning_message += warning_message
|
||||
|
||||
def append_critical(self, critical_message):
|
||||
self.critical_message += critical_message
|
||||
|
||||
def append_unknown(self, unknown_message):
|
||||
self.critical_message += critical_message
|
||||
def append_metrics(self, (performance_data, warning_message, critical_message)):
|
||||
self.unknown_message += unknown_message
|
||||
|
||||
def append_metrics(self, (performance_data,
|
||||
warning_message, critical_message)):
|
||||
self.performance_data += performance_data
|
||||
self.append_warning(warning_message)
|
||||
self.append_critical(critical_message)
|
||||
|
||||
|
||||
class JsonHelper:
|
||||
"""Perform simple comparison operations against values in a given JSON dict"""
|
||||
"""Perform simple comparison operations against values in a given
|
||||
JSON dict"""
|
||||
def __init__(self, json_data, separator):
|
||||
self.data = json_data
|
||||
self.separator = separator
|
||||
@@ -81,7 +96,8 @@ class JsonHelper:
|
||||
|
||||
def getSubArrayElement(self, key, data):
|
||||
subElemKey = key[:key.find(self.arrayOpener)]
|
||||
index = int(key[key.find(self.arrayOpener) + 1:key.find(self.arrayCloser)])
|
||||
index = int(key[key.find(self.arrayOpener) +
|
||||
1:key.find(self.arrayCloser)])
|
||||
remainingKey = key[key.find(self.arrayCloser + self.separator) + 2:]
|
||||
if key.find(self.arrayCloser + self.separator) == -1:
|
||||
remainingKey = key[key.find(self.arrayCloser) + 1:]
|
||||
@@ -96,21 +112,36 @@ class JsonHelper:
|
||||
else:
|
||||
return (None, 'not_found')
|
||||
|
||||
def equals(self, key, value): return self.exists(key) and str(self.get(key)) in value.split(':')
|
||||
def lte(self, key, value): return self.exists(key) and float(self.get(key)) <= float(value)
|
||||
def lt(self, key, value): return self.exists(key) and float(self.get(key)) < float(value)
|
||||
def gte(self, key, value): return self.exists(key) and float(self.get(key)) >= float(value)
|
||||
def gt(self, key, value): return self.exists(key) and float(self.get(key)) > float(value)
|
||||
def exists(self, key): return (self.get(key) != (None, 'not_found'))
|
||||
def equals(self, key, value):
|
||||
return self.exists(key) and \
|
||||
str(self.get(key)) in value.split(':')
|
||||
|
||||
def lte(self, key, value):
|
||||
return self.exists(key) and float(self.get(key)) <= float(value)
|
||||
|
||||
def lt(self, key, value):
|
||||
return self.exists(key) and float(self.get(key)) < float(value)
|
||||
|
||||
def gte(self, key, value):
|
||||
return self.exists(key) and float(self.get(key)) >= float(value)
|
||||
|
||||
def gt(self, key, value):
|
||||
return self.exists(key) and float(self.get(key)) > float(value)
|
||||
|
||||
def exists(self, key):
|
||||
return (self.get(key) != (None, 'not_found'))
|
||||
|
||||
def get(self, key, temp_data=''):
|
||||
"""Can navigate nested json keys with a dot format (Element.Key.NestedKey). Returns (None, 'not_found') if not found"""
|
||||
"""Can navigate nested json keys with a dot format
|
||||
(Element.Key.NestedKey). Returns (None, 'not_found') if not found"""
|
||||
if temp_data:
|
||||
data = temp_data
|
||||
else:
|
||||
data = self.data
|
||||
if len(key) <= 0:
|
||||
return data
|
||||
if key.find(self.separator) != -1 and key.find(self.arrayOpener) != -1 :
|
||||
if key.find(self.separator) != -1 and \
|
||||
key.find(self.arrayOpener) != -1:
|
||||
if key.find(self.separator) < key.find(self.arrayOpener):
|
||||
return self.getSubElement(key, data)
|
||||
else:
|
||||
@@ -127,6 +158,29 @@ class JsonHelper:
|
||||
else:
|
||||
return (None, 'not_found')
|
||||
|
||||
def expandKey(self, key, keys):
|
||||
if '(*)' not in key:
|
||||
keys.append(key)
|
||||
return keys
|
||||
subElemKey = ''
|
||||
if key.find('(*)') > 0:
|
||||
subElemKey = key[:key.find('(*)')-1]
|
||||
remainingKey = key[key.find('(*)')+3:]
|
||||
elemData = self.get(subElemKey)
|
||||
if elemData is (None, 'not_found'):
|
||||
keys.append(key)
|
||||
return keys
|
||||
if subElemKey is not '':
|
||||
subElemKey = subElemKey + '.'
|
||||
for i in range(len(elemData)):
|
||||
newKey = subElemKey + '(' + str(i) + ')' + remainingKey
|
||||
newKeys = self.expandKey(newKey, [])
|
||||
for j in newKeys:
|
||||
keys.append(j)
|
||||
|
||||
return keys
|
||||
|
||||
|
||||
def _getKeyAlias(original_key):
|
||||
key = original_key
|
||||
alias = original_key
|
||||
@@ -136,22 +190,51 @@ def _getKeyAlias(original_key):
|
||||
key, alias = keys
|
||||
return key, alias
|
||||
|
||||
|
||||
class JsonRuleProcessor:
|
||||
"""Perform checks and gather values from a JSON dict given rules and metrics definitions"""
|
||||
"""Perform checks and gather values from a JSON dict given rules
|
||||
and metrics definitions"""
|
||||
def __init__(self, json_data, rules_args):
|
||||
self.data = json_data
|
||||
self.rules = rules_args
|
||||
separator = '.'
|
||||
if self.rules.separator: separator = self.rules.separator
|
||||
if self.rules.separator:
|
||||
separator = self.rules.separator
|
||||
self.helper = JsonHelper(self.data, separator)
|
||||
debugPrint(rules_args.debug, "rules:%s" % rules_args)
|
||||
debugPrint(rules_args.debug, "separator:%s" % separator)
|
||||
self.metric_list = self.expandKeys(self.rules.metric_list)
|
||||
self.key_threshold_warning = self.expandKeys(
|
||||
self.rules.key_threshold_warning)
|
||||
self.key_threshold_critical = self.expandKeys(
|
||||
self.rules.key_threshold_critical)
|
||||
self.key_value_list = self.expandKeys(self.rules.key_value_list)
|
||||
self.key_value_list_not = self.expandKeys(
|
||||
self.rules.key_value_list_not)
|
||||
self.key_list = self.expandKeys(self.rules.key_list)
|
||||
self.key_value_list_critical = self.expandKeys(
|
||||
self.rules.key_value_list_critical)
|
||||
self.key_value_list_not_critical = self.expandKeys(
|
||||
self.rules.key_value_list_not_critical)
|
||||
self.key_list_critical = self.expandKeys(self.rules.key_list_critical)
|
||||
self.key_value_list_unknown = self.expandKeys(
|
||||
self.rules.key_value_list_unknown)
|
||||
|
||||
def expandKeys(self, src):
|
||||
if src is None:
|
||||
return
|
||||
dest = []
|
||||
for key in src:
|
||||
newKeys = self.helper.expandKey(key, [])
|
||||
for k in newKeys:
|
||||
dest.append(k)
|
||||
return dest
|
||||
|
||||
def checkExists(self, exists_list):
|
||||
failure = ''
|
||||
for k in exists_list:
|
||||
key, alias = _getKeyAlias(k)
|
||||
if (self.helper.exists(key) == False):
|
||||
if (self.helper.exists(key) is False):
|
||||
failure += " Key %s did not exist." % alias
|
||||
return failure
|
||||
|
||||
@@ -161,7 +244,18 @@ class JsonRuleProcessor:
|
||||
k, v = kv.split(',')
|
||||
key, alias = _getKeyAlias(k)
|
||||
if (self.helper.equals(key, v) == False):
|
||||
failure += " Value for key %s did not match %s." % (alias, v)
|
||||
failure += " Key %s mismatch. %s != %s" % (alias, v,
|
||||
self.helper.get(key))
|
||||
return failure
|
||||
|
||||
def checkNonEquality(self, equality_list):
|
||||
failure = ''
|
||||
for kv in equality_list:
|
||||
k, v = kv.split(',')
|
||||
key, alias = _getKeyAlias(k)
|
||||
if (self.helper.equals(key, v) == True):
|
||||
failure += " Key %s match found. %s == %s" % (alias, v,
|
||||
self.helper.get(key))
|
||||
return failure
|
||||
|
||||
def checkThreshold(self, key, alias, r):
|
||||
@@ -181,19 +275,28 @@ class JsonRuleProcessor:
|
||||
end = vals[1]
|
||||
if(start == '~'):
|
||||
if (invert and self.helper.lte(key, end)):
|
||||
failure += " Value for key %s was less than or equal to %s." % (alias, end)
|
||||
failure += " Value (%s) for key %s was less than or equal to %s." % \
|
||||
(self.helper.get(key), alias, end)
|
||||
elif (not invert and self.helper.gt(key, end)):
|
||||
failure += " Value for key %s was greater than %s." % (alias, end)
|
||||
failure += " Value (%s) for key %s was greater than %s." % \
|
||||
(self.helper.get(key), alias, end)
|
||||
elif(end == 'infinity'):
|
||||
if (invert and self.helper.gte(key, start)):
|
||||
failure += " Value for key %s was greater than or equal to %s." % (alias, start)
|
||||
failure += " Value (%s) for key %s was greater than or equal to %s." % \
|
||||
(self.helper.get(key), alias, start)
|
||||
elif (not invert and self.helper.lt(key, start)):
|
||||
failure += " Value for key %s was less than %s." % (alias, start)
|
||||
failure += " Value (%s) for key %s was less than %s." % \
|
||||
(self.helper.get(key), alias, start)
|
||||
else:
|
||||
if (invert and self.helper.gte(key, start) and self.helper.lte(key, end)):
|
||||
failure += " Value for key %s was inside the range %s:%s." % (alias, start, end)
|
||||
elif (not invert and (self.helper.lt(key, start) or self.helper.gt(key, end))):
|
||||
failure += " Value for key %s was outside the range %s:%s." % (alias, start, end)
|
||||
if (invert and self.helper.gte(key, start) and
|
||||
self.helper.lte(key, end)):
|
||||
failure += " Value (%s) for key %s was inside the range %s:%s." % \
|
||||
(self.helper.get(key), alias, start, end)
|
||||
elif (not invert and (self.helper.lt(key, start) or
|
||||
self.helper.gt(key, end))):
|
||||
failure += " Value (%s) for key %s was outside the range %s:%s." % \
|
||||
(self.helper.get(key), alias, start, end)
|
||||
|
||||
return failure
|
||||
|
||||
def checkThresholds(self, threshold_list):
|
||||
@@ -206,31 +309,42 @@ class JsonRuleProcessor:
|
||||
|
||||
def checkWarning(self):
|
||||
failure = ''
|
||||
if self.rules.key_threshold_warning != None:
|
||||
failure += self.checkThresholds(self.rules.key_threshold_warning)
|
||||
if self.rules.key_value_list != None:
|
||||
failure += self.checkEquality(self.rules.key_value_list)
|
||||
if self.rules.key_list != None:
|
||||
failure += self.checkExists(self.rules.key_list)
|
||||
if self.key_threshold_warning is not None:
|
||||
failure += self.checkThresholds(self.key_threshold_warning)
|
||||
if self.key_value_list is not None:
|
||||
failure += self.checkEquality(self.key_value_list)
|
||||
if self.key_value_list_not is not None:
|
||||
failure += self.checkNonEquality(self.key_value_list_not)
|
||||
if self.key_list is not None:
|
||||
failure += self.checkExists(self.key_list)
|
||||
return failure
|
||||
|
||||
def checkCritical(self):
|
||||
failure = ''
|
||||
if self.rules.key_threshold_critical != None:
|
||||
failure += self.checkThresholds(self.rules.key_threshold_critical)
|
||||
if self.rules.key_value_list_critical != None:
|
||||
failure += self.checkEquality(self.rules.key_value_list_critical)
|
||||
if self.rules.key_list_critical != None:
|
||||
failure += self.checkExists(self.rules.key_list_critical)
|
||||
if self.key_threshold_critical is not None:
|
||||
failure += self.checkThresholds(self.key_threshold_critical)
|
||||
if self.key_value_list_critical is not None:
|
||||
failure += self.checkEquality(self.key_value_list_critical)
|
||||
if self.key_value_list_not_critical is not None:
|
||||
failure += self.checkNonEquality(self.key_value_list_not_critical)
|
||||
if self.key_list_critical is not None:
|
||||
failure += self.checkExists(self.key_list_critical)
|
||||
return failure
|
||||
|
||||
def checkUnknown(self):
|
||||
unknown = ''
|
||||
if self.key_value_list_unknown is not None:
|
||||
unknown += self.checkEquality(self.key_value_list_unknown)
|
||||
return unknown
|
||||
|
||||
def checkMetrics(self):
|
||||
"""Return a Nagios specific performance metrics string given keys and parameter definitions"""
|
||||
"""Return a Nagios specific performance metrics string given keys
|
||||
and parameter definitions"""
|
||||
metrics = ''
|
||||
warning = ''
|
||||
critical = ''
|
||||
if self.rules.metric_list != None:
|
||||
for metric in self.rules.metric_list:
|
||||
if self.metric_list is not None:
|
||||
for metric in self.metric_list:
|
||||
key = metric
|
||||
minimum = maximum = warn_range = crit_range = None
|
||||
uom = ''
|
||||
@@ -241,91 +355,191 @@ class JsonRuleProcessor:
|
||||
if len(vals) == 4:
|
||||
key, uom, warn_range, crit_range = vals
|
||||
if len(vals) == 6:
|
||||
key,uom,warn_range,crit_range,minimum,maximum = vals
|
||||
key, uom, warn_range, crit_range, \
|
||||
minimum, maximum = vals
|
||||
key, alias = _getKeyAlias(key)
|
||||
if self.helper.exists(key):
|
||||
metrics += "'%s'=%s" % (alias, self.helper.get(key))
|
||||
if uom: metrics += uom
|
||||
if warn_range != None:
|
||||
if uom:
|
||||
metrics += uom
|
||||
if warn_range is not None:
|
||||
warning += self.checkThreshold(key, alias, warn_range)
|
||||
metrics += ";%s" % warn_range
|
||||
if crit_range != None:
|
||||
if crit_range is not None:
|
||||
critical += self.checkThreshold(key, alias, crit_range)
|
||||
metrics += ";%s" % crit_range
|
||||
if minimum != None:
|
||||
critical += self.checkThreshold(key, alias, minimum + ':')
|
||||
if minimum is not None:
|
||||
critical += self.checkThreshold(key, alias, minimum +
|
||||
':')
|
||||
metrics += ";%s" % minimum
|
||||
if maximum != None:
|
||||
critical += self.checkThreshold(key, alias, '~:' + maximum)
|
||||
if maximum is not None:
|
||||
critical += self.checkThreshold(key, alias, '~:' +
|
||||
maximum)
|
||||
metrics += ";%s" % maximum
|
||||
metrics += ' '
|
||||
return ("%s" % metrics, warning, critical)
|
||||
|
||||
def parseArgs():
|
||||
parser = argparse.ArgumentParser(description=
|
||||
'Nagios plugin which checks json values from a given endpoint against argument specified rules\
|
||||
and determines the status and performance data for that service')
|
||||
|
||||
# parser.add_argument('-v', '--verbose', action='store_true', help='Verbose Output')
|
||||
parser.add_argument('-d', '--debug', action='store_true', help='Debug mode.')
|
||||
parser.add_argument('-s', '--ssl', action='store_true', help='HTTPS mode.')
|
||||
parser.add_argument('-H', '--host', dest='host', required=True, help='Host.')
|
||||
def parseArgs():
|
||||
parser = argparse.ArgumentParser(
|
||||
description = plugin_description + '\n\nVersion: %s (%s)'
|
||||
%(__version__, __version_date__),
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
|
||||
parser.add_argument('-d', '--debug', action='store_true',
|
||||
help='debug mode')
|
||||
parser.add_argument('-s', '--ssl', action='store_true',
|
||||
help='use TLS to connect to remote host')
|
||||
parser.add_argument('-H', '--host', dest='host',
|
||||
required=not ('-V' in sys.argv or '--version' in sys.argv),
|
||||
help='remote host to query')
|
||||
parser.add_argument('-k', '--insecure', action='store_true',
|
||||
help='do not check server SSL certificate')
|
||||
parser.add_argument('-V', '--version', action='store_true',
|
||||
help='print version of this plugin')
|
||||
parser.add_argument('--cacert',
|
||||
dest='cacert', help='SSL CA certificate')
|
||||
parser.add_argument('--cert',
|
||||
dest='cert', help='SSL client certificate')
|
||||
parser.add_argument('--key', dest='key',
|
||||
help='SSL client key ( if not bundled into the cert )')
|
||||
parser.add_argument('-P', '--port', dest='port', help='TCP port')
|
||||
parser.add_argument('-p', '--path', dest='path', help='Path.')
|
||||
parser.add_argument('-t', '--timeout', type=int, help='Connection timeout (seconds)')
|
||||
parser.add_argument('-B', '--basic-auth', dest='auth', help='Basic auth string "username:password"')
|
||||
parser.add_argument('-D', '--data', dest='data', help='The http payload to send as a POST')
|
||||
parser.add_argument('-p', '--path', dest='path', help='Path')
|
||||
parser.add_argument('-t', '--timeout', type=int,
|
||||
help='Connection timeout (seconds)')
|
||||
parser.add_argument('-B', '--basic-auth', dest='auth',
|
||||
help='Basic auth string "username:password"')
|
||||
parser.add_argument('-D', '--data', dest='data',
|
||||
help='The http payload to send as a POST')
|
||||
parser.add_argument('-A', '--headers', dest='headers',
|
||||
help='The http headers in JSON format.')
|
||||
parser.add_argument('-f', '--field_separator', dest='separator',
|
||||
help='Json Field separator, defaults to "." ; Select element in an array with "(" ")"')
|
||||
parser.add_argument('-w', '--warning', dest='key_threshold_warning', nargs='*',
|
||||
help='Warning threshold for these values (key1[>alias],WarnRange key2[>alias],WarnRange). WarnRange is in the format [@]start:end, more information at nagios-plugins.org/doc/guidelines.html.')
|
||||
parser.add_argument('-c', '--critical', dest='key_threshold_critical', nargs='*',
|
||||
help='Critical threshold for these values (key1[>alias],CriticalRange key2[>alias],CriticalRange. CriticalRange is in the format [@]start:end, more information at nagios-plugins.org/doc/guidelines.html.')
|
||||
help='''JSON Field separator, defaults to ".";
|
||||
Select element in an array with "(" ")"''')
|
||||
parser.add_argument('-w', '--warning', dest='key_threshold_warning',
|
||||
nargs='*',
|
||||
help='''Warning threshold for these values
|
||||
(key1[>alias],WarnRange key2[>alias],WarnRange).
|
||||
WarnRange is in the format [@]start:end, more
|
||||
information at
|
||||
nagios-plugins.org/doc/guidelines.html.''')
|
||||
parser.add_argument('-c', '--critical', dest='key_threshold_critical',
|
||||
nargs='*',
|
||||
help='''Critical threshold for these values
|
||||
(key1[>alias],CriticalRange key2[>alias],CriticalRange.
|
||||
CriticalRange is in the format [@]start:end, more
|
||||
information at
|
||||
nagios-plugins.org/doc/guidelines.html.''')
|
||||
parser.add_argument('-e', '--key_exists', dest='key_list', nargs='*',
|
||||
help='Checks existence of these keys to determine status. Return warning if key is not present.')
|
||||
parser.add_argument('-E', '--key_exists_critical', dest='key_list_critical', nargs='*',
|
||||
help='Same as -e but return critical if key is not present.')
|
||||
help='''Checks existence of these keys to determine
|
||||
status. Return warning if key is not present.''')
|
||||
parser.add_argument('-E', '--key_exists_critical',
|
||||
dest='key_list_critical',
|
||||
nargs='*',
|
||||
help='''Same as -e but return critical if key is
|
||||
not present.''')
|
||||
parser.add_argument('-q', '--key_equals', dest='key_value_list', nargs='*',
|
||||
help='Checks equality of these keys and values (key[>alias],value key2,value2) to determine status.\
|
||||
Multiple key values can be delimited with colon (key,value1:value2). Return warning if equality check fails')
|
||||
parser.add_argument('-Q', '--key_equals_critical', dest='key_value_list_critical', nargs='*',
|
||||
help='Same as -q but return critical if equality check fails.')
|
||||
help='''Checks equality of these keys and values
|
||||
(key[>alias],value key2,value2) to determine status.
|
||||
Multiple key values can be delimited with colon
|
||||
(key,value1:value2). Return warning if equality
|
||||
check fails''')
|
||||
parser.add_argument('-Q', '--key_equals_critical',
|
||||
dest='key_value_list_critical', nargs='*',
|
||||
help='''Same as -q but return critical if
|
||||
equality check fails.''')
|
||||
parser.add_argument('-u', '--key_equals_unknown',
|
||||
dest='key_value_list_unknown', nargs='*',
|
||||
help='''Same as -q but return unknown if
|
||||
equality check fails.''')
|
||||
parser.add_argument('-y', '--key_not_equals',
|
||||
dest='key_value_list_not', nargs='*',
|
||||
help='''Checks equality of these keys and values
|
||||
(key[>alias],value key2,value2) to determine status.
|
||||
Multiple key values can be delimited with colon
|
||||
(key,value1:value2). Return warning if equality
|
||||
check succeeds''')
|
||||
parser.add_argument('-Y', '--key_not_equals_critical',
|
||||
dest='key_value_list_not_critical', nargs='*',
|
||||
help='''Same as -q but return critical if equality
|
||||
check succeeds.''')
|
||||
parser.add_argument('-m', '--key_metric', dest='metric_list', nargs='*',
|
||||
help='Gathers the values of these keys (key[>alias],UnitOfMeasure,WarnRange,CriticalRange,Min,Max) for Nagios performance data.\
|
||||
More information about Range format and units of measure for nagios can be found at nagios-plugins.org/doc/guidelines.html\
|
||||
Additional formats for this parameter are: (key[>alias]), (key[>alias],UnitOfMeasure), (key[>alias],UnitOfMeasure,WarnRange,CriticalRange).')
|
||||
help='''Gathers the values of these keys (key[>alias],
|
||||
UnitOfMeasure,WarnRange,CriticalRange,Min,Max) for
|
||||
Nagios performance data. More information about Range
|
||||
format and units of measure for nagios can be found at
|
||||
nagios-plugins.org/doc/guidelines.html
|
||||
Additional formats for this parameter are:
|
||||
(key[>alias]), (key[>alias],UnitOfMeasure),
|
||||
(key[>alias],UnitOfMeasure,WarnRange,
|
||||
CriticalRange).''')
|
||||
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def debugPrint(debug_flag, message, pretty_flag=False):
|
||||
if debug_flag:
|
||||
if pretty_flag:
|
||||
pprint(message)
|
||||
else:
|
||||
print message
|
||||
print(message)
|
||||
|
||||
if __name__ == "__main__" and len(sys.argv) >= 2 and sys.argv[1] == 'UnitTest':
|
||||
import unittest
|
||||
|
||||
class RulesHelper:
|
||||
separator = '.'
|
||||
debug = False
|
||||
key_threshold_warning,key_value_list,key_list,key_threshold_critical,key_value_list_critical,key_list_critical,metric_list = None, None, None, None, None, None, None
|
||||
key_threshold_warning = None
|
||||
key_value_list = None
|
||||
key_value_list_not = None
|
||||
key_list = None
|
||||
key_threshold_critical = None
|
||||
key_value_list_critical = None
|
||||
key_value_list_not_critical = None
|
||||
key_value_list_unknown = None
|
||||
key_list_critical = None
|
||||
metric_list = None
|
||||
|
||||
def dash_m(self, data):
|
||||
self.metric_list = data; return self
|
||||
self.metric_list = data
|
||||
return self
|
||||
|
||||
def dash_e(self, data):
|
||||
self.key_list = data; return self
|
||||
self.key_list = data
|
||||
return self
|
||||
|
||||
def dash_E(self, data):
|
||||
self.key_list_critical = data; return self
|
||||
self.key_list_critical = data
|
||||
return self
|
||||
|
||||
def dash_q(self, data):
|
||||
self.key_value_list = data; return self
|
||||
self.key_value_list = data
|
||||
return self
|
||||
|
||||
def dash_Q(self, data):
|
||||
self.key_value_list_critical = data; return self
|
||||
self.key_value_list_critical = data
|
||||
return self
|
||||
|
||||
def dash_y(self, data):
|
||||
self.key_value_list_not = data
|
||||
return self
|
||||
|
||||
def dash_Y(self, data):
|
||||
self.key_value_list_not_critical = data
|
||||
return self
|
||||
|
||||
def dash_w(self, data):
|
||||
self.key_threshold_warning = data; return self
|
||||
self.key_threshold_warning = data
|
||||
return self
|
||||
|
||||
def dash_c(self, data):
|
||||
self.key_threshold_critical = data; return self
|
||||
self.key_threshold_critical = data
|
||||
return self
|
||||
|
||||
class UnitTest(unittest.TestCase):
|
||||
rules = RulesHelper()
|
||||
|
||||
def check_data(self, args, jsondata, code):
|
||||
data = json.loads(jsondata)
|
||||
nagios = NagiosHelper()
|
||||
@@ -334,61 +548,134 @@ if __name__ == "__main__" and len(sys.argv) >= 2 and sys.argv[1] == 'UnitTest':
|
||||
nagios.append_critical(processor.checkCritical())
|
||||
nagios.append_metrics(processor.checkMetrics())
|
||||
self.assertEqual(code, nagios.getCode())
|
||||
|
||||
def test_metrics(self):
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:4,1:5']), '{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:5,1:4']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:5,1:5,6,10']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:5,1:5,1,4']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,s,@1:4,@6:10,1,10']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:4,1:5']),
|
||||
'{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:5,1:4']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:5,1:5,6,10']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,,1:5,1:5,1,4']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['metric,s,@1:4,@6:10,1,10']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_m(['(*).value,s,1:5,1:5']),
|
||||
'[{"value": 5},{"value": 100}]', CRITICAL_CODE)
|
||||
|
||||
def test_exists(self):
|
||||
self.check_data(RulesHelper().dash_e(['nothere']), '{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_E(['nothere']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_e(['metric']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_e(['nothere']),
|
||||
'{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_E(['nothere']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_e(['metric']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
|
||||
def test_equality(self):
|
||||
self.check_data(RulesHelper().dash_q(['metric,6']), '{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_Q(['metric,6']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_q(['metric,5']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_q(['metric,6']),
|
||||
'{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_Q(['metric,6']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_q(['metric,5']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
|
||||
def test_non_equality(self):
|
||||
self.check_data(RulesHelper().dash_y(['metric,6']),
|
||||
'{"metric": 6}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_Y(['metric,6']),
|
||||
'{"metric": 6}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_y(['metric,5']),
|
||||
'{"metric": 6}', OK_CODE)
|
||||
|
||||
def test_warning_thresholds(self):
|
||||
self.check_data(RulesHelper().dash_w(['metric,5']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,5:']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,~:5']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,1:5']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5']), '{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5:']), '{"metric": 4}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@~:5']), '{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@1:5']), '{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,5']), '{"metric": 6}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,5:']), '{"metric": 4}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,~:5']), '{"metric": 6}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,1:5']), '{"metric": 6}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5']), '{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5:']), '{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@~:5']), '{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@1:5']), '{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,5']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,5:']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,~:5']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,1:5']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5']),
|
||||
'{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5:']),
|
||||
'{"metric": 4}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@~:5']),
|
||||
'{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@1:5']),
|
||||
'{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,5']),
|
||||
'{"metric": 6}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,5:']),
|
||||
'{"metric": 4}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,~:5']),
|
||||
'{"metric": 6}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,1:5']),
|
||||
'{"metric": 6}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5']),
|
||||
'{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@5:']),
|
||||
'{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@~:5']),
|
||||
'{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['metric,@1:5']),
|
||||
'{"metric": 5}', WARNING_CODE)
|
||||
self.check_data(RulesHelper().dash_w(['(*).value,@1:5']),
|
||||
'[{"value": 5},{"value": 1000}]', WARNING_CODE)
|
||||
|
||||
def test_critical_thresholds(self):
|
||||
self.check_data(RulesHelper().dash_c(['metric,5']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,5:']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,~:5']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,1:5']), '{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5']), '{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5:']), '{"metric": 4}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@~:5']), '{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@1:5']), '{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,5']), '{"metric": 6}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,5:']), '{"metric": 4}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,~:5']), '{"metric": 6}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,1:5']), '{"metric": 6}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5:']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@~:5']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@1:5']), '{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,5']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,5:']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,~:5']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,1:5']),
|
||||
'{"metric": 5}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5']),
|
||||
'{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5:']),
|
||||
'{"metric": 4}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@~:5']),
|
||||
'{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@1:5']),
|
||||
'{"metric": 6}', OK_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,5']),
|
||||
'{"metric": 6}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,5:']),
|
||||
'{"metric": 4}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,~:5']),
|
||||
'{"metric": 6}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,1:5']),
|
||||
'{"metric": 6}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@5:']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@~:5']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['metric,@1:5']),
|
||||
'{"metric": 5}', CRITICAL_CODE)
|
||||
self.check_data(RulesHelper().dash_c(['(*).value,@1:5']),
|
||||
'[{"value": 5},{"value": 1000}]', CRITICAL_CODE)
|
||||
|
||||
def test_separator(self):
|
||||
rules = RulesHelper()
|
||||
rules.separator = '_'
|
||||
self.check_data(
|
||||
rules.dash_q(['(0)_gauges_jvm.buffers.direct.capacity(1)_value,1234']),
|
||||
'[{ "gauges": { "jvm.buffers.direct.capacity": [{"value": 215415},{"value": 1234}]}}]',
|
||||
rules.dash_q(
|
||||
['(0)_gauges_jvm.buffers.direct.capacity(1)_value,1234']),
|
||||
'''[{ "gauges": { "jvm.buffers.direct.capacity": [
|
||||
{"value": 215415},{"value": 1234}]}}]''',
|
||||
OK_CODE)
|
||||
self.check_data(
|
||||
rules.dash_q(
|
||||
['(*)_gauges_jvm.buffers.direct.capacity(1)_value,1234']),
|
||||
'''[{ "gauges": { "jvm.buffers.direct.capacity": [
|
||||
{"value": 215415},{"value": 1234}]}},
|
||||
{ "gauges": { "jvm.buffers.direct.capacity": [
|
||||
{"value": 215415},{"value": 1235}]}}]''',
|
||||
WARNING_CODE)
|
||||
unittest.main()
|
||||
exit(0)
|
||||
|
||||
@@ -396,34 +683,89 @@ if __name__ == "__main__" and len(sys.argv) >= 2 and sys.argv[1] == 'UnitTest':
|
||||
if __name__ == "__main__":
|
||||
args = parseArgs()
|
||||
nagios = NagiosHelper()
|
||||
if args.version:
|
||||
print('Version: %s - Date: %s' % (__version__, __version_date__))
|
||||
exit(0)
|
||||
|
||||
if args.ssl:
|
||||
url = "https://%s" % args.host
|
||||
|
||||
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
|
||||
context.options |= ssl.OP_NO_SSLv2
|
||||
context.options |= ssl.OP_NO_SSLv3
|
||||
|
||||
if args.insecure:
|
||||
context.verify_mode = ssl.CERT_NONE
|
||||
else:
|
||||
context.verify_mode = ssl.CERT_OPTIONAL
|
||||
if args.cacert:
|
||||
try:
|
||||
context.load_verify_locations(args.cacert)
|
||||
except ssl.SSLError:
|
||||
nagios.append_unknown(
|
||||
''' Error loading SSL CA cert "%s"!'''
|
||||
% args.cacert)
|
||||
|
||||
if args.cert:
|
||||
try:
|
||||
context.load_cert_chain(args.cert,keyfile=args.key)
|
||||
except ssl.SSLError:
|
||||
if args.key:
|
||||
nagios.append_unknown(
|
||||
''' Error loading SSL cert. Make sure key "%s" belongs to cert "%s"!'''
|
||||
% (args.key, args.cert))
|
||||
else:
|
||||
nagios.append_unknown(
|
||||
''' Error loading SSL cert. Make sure "%s" contains the key as well!'''
|
||||
% (args.cert))
|
||||
|
||||
if nagios.getCode() != OK_CODE:
|
||||
print(nagios.getMessage())
|
||||
exit(nagios.getCode())
|
||||
|
||||
else:
|
||||
url = "http://%s" % args.host
|
||||
if args.port: url += ":%s" % args.port
|
||||
if args.path: url += "/%s" % args.path
|
||||
if args.port:
|
||||
url += ":%s" % args.port
|
||||
if args.path:
|
||||
url += "/%s" % args.path
|
||||
debugPrint(args.debug, "url:%s" % url)
|
||||
# Attempt to reach the endpoint
|
||||
json_data = ''
|
||||
try:
|
||||
req = urllib2.Request(url)
|
||||
req.add_header("User-Agent", "check_http_json")
|
||||
if args.auth:
|
||||
base64str = base64.encodestring(args.auth).replace('\n', '')
|
||||
req.add_header('Authorization', 'Basic %s' % base64str)
|
||||
if args.headers:
|
||||
headers = json.loads(args.headers)
|
||||
debugPrint(args.debug, "Headers:\n %s" % headers)
|
||||
for header in headers:
|
||||
req.add_header(header, headers[header])
|
||||
if args.timeout and args.data:
|
||||
response = urllib2.urlopen(req, timeout=args.timeout, data=args.data)
|
||||
response = urllib2.urlopen(req, timeout=args.timeout,
|
||||
data=args.data, context=context)
|
||||
elif args.timeout:
|
||||
response = urllib2.urlopen(req, timeout=args.timeout)
|
||||
response = urllib2.urlopen(req, timeout=args.timeout,
|
||||
context=context)
|
||||
elif args.data:
|
||||
response = urllib2.urlopen(req, data=args.data)
|
||||
response = urllib2.urlopen(req, data=args.data, context=context)
|
||||
else:
|
||||
response = urllib2.urlopen(req)
|
||||
response = urllib2.urlopen(req, context=context)
|
||||
|
||||
json_data = response.read()
|
||||
|
||||
except HTTPError as e:
|
||||
nagios.append_unknown(" HTTPError[%s], url:%s" % (str(e.code), url))
|
||||
except URLError as e:
|
||||
nagios.append_critical(" URLError[%s], url:%s" % (str(e.reason), url))
|
||||
|
||||
try:
|
||||
data = json.loads(json_data)
|
||||
except ValueError as e:
|
||||
nagios.append_unknown(" Parser error: %s" % str(e))
|
||||
|
||||
else:
|
||||
jsondata = response.read()
|
||||
data = json.loads(jsondata)
|
||||
debugPrint(args.debug, 'json:')
|
||||
debugPrint(args.debug, data, True)
|
||||
# Apply rules to returned JSON data
|
||||
@@ -431,6 +773,10 @@ if __name__ == "__main__":
|
||||
nagios.append_warning(processor.checkWarning())
|
||||
nagios.append_critical(processor.checkCritical())
|
||||
nagios.append_metrics(processor.checkMetrics())
|
||||
nagios.append_unknown(processor.checkUnknown())
|
||||
|
||||
# Print Nagios specific string and exit appropriately
|
||||
print nagios.getMessage()
|
||||
print(nagios.getMessage())
|
||||
exit(nagios.getCode())
|
||||
|
||||
#EOF
|
||||
|
||||
115
docs/DOCKER.md
Normal file
115
docs/DOCKER.md
Normal file
@@ -0,0 +1,115 @@
|
||||
### Docker Info Example Plugin
|
||||
|
||||
#### Description
|
||||
|
||||
Let's say we want to use `check_http_json.py` to read from Docker's `/info` HTTP API endpoint with the following parameters:
|
||||
|
||||
##### Connection information
|
||||
|
||||
* Host = 127.0.0.1:4243
|
||||
* Path = /info
|
||||
|
||||
##### Rules for "aliveness"
|
||||
|
||||
* Verify that the key `Containers` exists in the outputted JSON
|
||||
* Verify that the key `IPv4Forwarding` has a value of `1`
|
||||
* Verify that the key `Debug` has a value less than or equal to `2`
|
||||
* Verify that the key `Images` has a value greater than or equal to `1`
|
||||
* If any of these criteria are not met, report a WARNING to Nagios
|
||||
|
||||
##### Gather Metrics
|
||||
|
||||
* Report value of the key `Containers` with a MinValue of 0 and a MaxValue of 1000 as performance data
|
||||
* Report value of the key `Images` as performance data
|
||||
* Report value of the key `NEventsListener` as performance data
|
||||
* Report value of the key `NFd` as performance data
|
||||
* Report value of the key `NGoroutines` as performance data
|
||||
* Report value of the key `SwapLimit` as performance data
|
||||
|
||||
#### Service Definition
|
||||
|
||||
`localhost.cfg`
|
||||
|
||||
```
|
||||
define service {
|
||||
use local-service
|
||||
host_name localhost
|
||||
service_description Docker info status checker
|
||||
check_command check_docker
|
||||
}
|
||||
```
|
||||
|
||||
#### Command Definition with Arguments
|
||||
|
||||
`commands.cfg`
|
||||
|
||||
```
|
||||
define command{
|
||||
command_name check_docker
|
||||
command_line /usr/bin/python /usr/local/nagios/libexec/plugins/check_http_json.py -H 127.0.0.1:4243 -p info -e Containers -q IPv4Forwarding,1 -w Debug,2:2 -c Images,1:1 -m Containers,0:250,0:500,0,1000 Images NEventsListener NFd NGoroutines SwapLimit
|
||||
}
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
OK: Status OK.|'Containers'=1;0;1000 'Images'=11;0;0 'NEventsListener'=3;0;0 'NFd'=10;0;0 'NGoroutines'=14;0;0 'SwapLimit'=1;0;0
|
||||
```
|
||||
|
||||
### Docker Container Monitor Example Plugin
|
||||
|
||||
`check_http_json.py` is generic enough to read and evaluate rules on any HTTP endpoint that returns JSON. In this example we'll get the status of a specific container using it's ID which camn be found by using the list containers endpoint (`curl http://127.0.0.1:4243/containers/json?all=1`).
|
||||
|
||||
##### Connection information
|
||||
|
||||
* Host = 127.0.0.1:4243
|
||||
* Path = /containers/2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615/json
|
||||
|
||||
##### Rules for "aliveness"
|
||||
|
||||
* Verify that the key `ID` exists and is equal to the value `2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615`
|
||||
* Verify that the key `State.Running` has a value of `True`
|
||||
|
||||
#### Service Definition
|
||||
|
||||
`localhost.cfg`
|
||||
|
||||
```
|
||||
define service {
|
||||
use local-service
|
||||
host_name localhost
|
||||
service_description Docker container liveness check
|
||||
check_command check_my_container
|
||||
}
|
||||
```
|
||||
|
||||
#### Command Definition with Arguments
|
||||
|
||||
`commands.cfg`
|
||||
|
||||
```
|
||||
define command{
|
||||
command_name check_my_container
|
||||
command_line /usr/bin/python /usr/local/nagios/libexec/plugins/check_http_json.py -H 127.0.0.1:4243 -p /containers/2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615/json -q ID,2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615 State.Running,True
|
||||
}
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
WARNING: Status check failed, reason: Value True for key State.Running did not match.
|
||||
```
|
||||
|
||||
The plugin threw a warning because the Container ID I used on my system has the following State object:
|
||||
|
||||
```
|
||||
u'State': {...
|
||||
u'Running': False,
|
||||
...
|
||||
```
|
||||
|
||||
If I change the command to have the parameter -q parameter `State.Running,False`, the output becomes:
|
||||
|
||||
```
|
||||
OK: Status OK.
|
||||
```
|
||||
227
docs/RIAK.md
Normal file
227
docs/RIAK.md
Normal file
@@ -0,0 +1,227 @@
|
||||
# Riak Stats Example
|
||||
|
||||
## Description
|
||||
|
||||
For this example we're going to use `check_http_json.py` as a pure CLI tool to read Riak's `/stats` endpoint
|
||||
|
||||
## Connection information
|
||||
|
||||
* Host = 127.0.0.1:8098
|
||||
* Path = /stats
|
||||
|
||||
## JSON Stats Data
|
||||
|
||||
* Full Riak HTTP Stats information can be found here: [http://docs.basho.com/riak/latest/dev/references/http/status/](http://docs.basho.com/riak/latest/dev/references/http/status/)
|
||||
* Information related to specific interesting stats can be found here: [http://docs.basho.com/riak/latest/ops/running/stats-and-monitoring/](http://docs.basho.com/riak/latest/ops/running/stats-and-monitoring/)
|
||||
|
||||
## Connectivity Check
|
||||
|
||||
* `ring_members`: We can use an existence check to monitor the number of ring members
|
||||
* `connected_nodes`: Similarly we can check the number of nodes that are in communication with this node, but this list will be empty in a 1 node cluster
|
||||
|
||||
#### Sample Command
|
||||
|
||||
For a single node dev "cluster", you might have a `ring_members` value like this:
|
||||
|
||||
```
|
||||
"ring_members": [
|
||||
"riak@127.0.0.1"
|
||||
],
|
||||
```
|
||||
|
||||
To validate that we have a single node, we can use this check:
|
||||
|
||||
```
|
||||
$ ./check_http_json.py -H localhost -P 8098 -p stats -E "ring_members(0)"
|
||||
OK: Status OK.
|
||||
```
|
||||
|
||||
If we were expecting at least 2 nodes in the cluster, we would use this check:
|
||||
|
||||
```
|
||||
$ ./check_http_json.py -H localhost -P 8098 -p stats -E "ring_members(1)"
|
||||
CRITICAL: Status CRITICAL. Key ring_members(1) did not exist.
|
||||
```
|
||||
|
||||
Obviously this fails because we only had a single `ring_member`. If we prefer to only get a warning instead of a critical for this check, we just use the correct flag:
|
||||
|
||||
```
|
||||
$ ./check_http_json.py -H localhost -P 8098 -p stats -e "ring_members(1)"
|
||||
WARNING: Status WARNING. Key ring_members(1) did not exist.
|
||||
```
|
||||
|
||||
## Gather Metrics
|
||||
|
||||
The thresholds for acceptable values for these metrics will vary from system to system, following are the stats we'll be checking:
|
||||
|
||||
### Throughput Metrics:
|
||||
|
||||
* `node_gets`
|
||||
* `node_puts`
|
||||
* `vnode_counter_update`
|
||||
* `vnode_set_update`
|
||||
* `vnode_map_update`
|
||||
* `search_query_throughput_one`
|
||||
* `search_index_throughtput_one`
|
||||
* `consistent_gets`
|
||||
* `consistent_puts`
|
||||
* `vnode_index_reads`
|
||||
|
||||
#### Sample Command
|
||||
|
||||
```
|
||||
./check_http_json.py -H localhost -P 8098 -p stats -m \
|
||||
"node_gets" \
|
||||
"node_puts" \
|
||||
"vnode_counter_update" \
|
||||
"vnode_set_update" \
|
||||
"vnode_map_update" \
|
||||
"search_query_throughput_one" \
|
||||
"search_index_throughtput_one" \
|
||||
"consistent_gets" \
|
||||
"consistent_puts" \
|
||||
"vnode_index_reads"
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
OK: Status OK.|'node_gets'=0 'node_puts'=0 'vnode_counter_update'=0 'vnode_set_update'=0 'vnode_map_update'=0 'search_query_throughput_one'=0 'consistent_gets'=0 'consistent_puts'=0 'vnode_index_reads'=0
|
||||
```
|
||||
|
||||
### Latency Metrics:
|
||||
|
||||
* `node_get_fsm_time_mean,_median,_95,_99,_100`
|
||||
* `node_put_fsm_time_mean,_median,_95,_99,_100`
|
||||
* `object_counter_merge_time_mean,_median,_95,_99,_100`
|
||||
* `object_set_merge_time_mean,_median,_95,_99,_100`
|
||||
* `object_map_merge_time_mean,_median,_95,_99,_100`
|
||||
* `search_query_latency_median,_min,_95,_99,_999`
|
||||
* `search_index_latency_median,_min,_95,_99,_999`
|
||||
* `consistent_get_time_mean,_median,_95,_99,_100`
|
||||
* `consistent_put_time_mean,_median,_95,_99,_100`
|
||||
|
||||
#### Sample Command
|
||||
|
||||
```
|
||||
./check_http_json.py -H localhost -P 8098 -p stats -m \
|
||||
"node_get_fsm_time_mean,,0:100,0:1000" \
|
||||
"node_get_fsm_time_median,,0:100,0:1000" \
|
||||
"node_get_fsm_time_95,,0:100,0:1000" \
|
||||
"node_get_fsm_time_99,,0:100,0:1000" \
|
||||
"node_get_fsm_time_100,,0:100,0:1000" \
|
||||
"node_put_fsm_time_mean,,0:100,0:1000" \
|
||||
"node_put_fsm_time_median,,0:100,0:1000" \
|
||||
"node_put_fsm_time_95,,0:100,0:1000" \
|
||||
"node_put_fsm_time_99,,0:100,0:1000" \
|
||||
"node_put_fsm_time_100,,0:100,0:1000" \
|
||||
"object_counter_merge_time_mean,,0:100,0:1000" \
|
||||
"object_counter_merge_time_median,,0:100,0:1000" \
|
||||
"object_counter_merge_time_95,,0:100,0:1000" \
|
||||
"object_counter_merge_time_99,,0:100,0:1000" \
|
||||
"object_counter_merge_time_100,,0:100,0:1000" \
|
||||
"object_set_merge_time_mean,,0:100,0:1000" \
|
||||
"object_set_merge_time_median,,0:100,0:1000" \
|
||||
"object_set_merge_time_95,,0:100,0:1000" \
|
||||
"object_set_merge_time_99,,0:100,0:1000" \
|
||||
"object_set_merge_time_100,,0:100,0:1000" \
|
||||
"object_map_merge_time_mean,,0:100,0:1000" \
|
||||
"object_map_merge_time_median,,0:100,0:1000" \
|
||||
"object_map_merge_time_95,,0:100,0:1000" \
|
||||
"object_map_merge_time_99,,0:100,0:1000" \
|
||||
"object_map_merge_time_100,,0:100,0:1000" \
|
||||
"consistent_get_time_mean,,0:100,0:1000" \
|
||||
"consistent_get_time_median,,0:100,0:1000" \
|
||||
"consistent_get_time_95,,0:100,0:1000" \
|
||||
"consistent_get_time_99,,0:100,0:1000" \
|
||||
"consistent_get_time_100,,0:100,0:1000" \
|
||||
"consistent_put_time_mean,,0:100,0:1000" \
|
||||
"consistent_put_time_median,,0:100,0:1000" \
|
||||
"consistent_put_time_95,,0:100,0:1000" \
|
||||
"consistent_put_time_99,,0:100,0:1000" \
|
||||
"consistent_put_time_100,,0:100,0:1000" \
|
||||
"search_query_latency_median,,0:100,0:1000" \
|
||||
"search_query_latency_min,,0:100,0:1000" \
|
||||
"search_query_latency_95,,0:100,0:1000" \
|
||||
"search_query_latency_99,,0:100,0:1000" \
|
||||
"search_query_latency_999,,0:100,0:1000" \
|
||||
"search_index_latency_median,,0:100,0:1000" \
|
||||
"search_index_latency_min,,0:100,0:1000" \
|
||||
"search_index_latency_95,,0:100,0:1000" \
|
||||
"search_index_latency_99,,0:100,0:1000" \
|
||||
"search_index_latency_999,,0:100,0:1000"
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
OK: Status OK.|'node_get_fsm_time_mean'=0;0:100;0:1000 'node_get_fsm_time_median'=0;0:100;0:1000 'node_get_fsm_time_95'=0;0:100;0:1000 'node_get_fsm_time_99'=0;0:100;0:1000 'node_get_fsm_time_100'=0;0:100;0:1000 'node_put_fsm_time_mean'=0;0:100;0:1000 'node_put_fsm_time_median'=0;0:100;0:1000 'node_put_fsm_time_95'=0;0:100;0:1000 'node_put_fsm_time_99'=0;0:100;0:1000 'node_put_fsm_time_100'=0;0:100;0:1000 'object_counter_merge_time_mean'=0;0:100;0:1000 'object_counter_merge_time_median'=0;0:100;0:1000 'object_counter_merge_time_95'=0;0:100;0:1000 'object_counter_merge_time_99'=0;0:100;0:1000 'object_counter_merge_time_100'=0;0:100;0:1000 'object_set_merge_time_mean'=0;0:100;0:1000 'object_set_merge_time_median'=0;0:100;0:1000 'object_set_merge_time_95'=0;0:100;0:1000 'object_set_merge_time_99'=0;0:100;0:1000 'object_set_merge_time_100'=0;0:100;0:1000 'object_map_merge_time_mean'=0;0:100;0:1000 'object_map_merge_time_median'=0;0:100;0:1000 'object_map_merge_time_95'=0;0:100;0:1000 'object_map_merge_time_99'=0;0:100;0:1000 'object_map_merge_time_100'=0;0:100;0:1000 'consistent_get_time_mean'=0;0:100;0:1000 'consistent_get_time_median'=0;0:100;0:1000 'consistent_get_time_95'=0;0:100;0:1000 'consistent_get_time_99'=0;0:100;0:1000 'consistent_get_time_100'=0;0:100;0:1000 'consistent_put_time_mean'=0;0:100;0:1000 'consistent_put_time_median'=0;0:100;0:1000 'consistent_put_time_95'=0;0:100;0:1000 'consistent_put_time_99'=0;0:100;0:1000 'consistent_put_time_100'=0;0:100;0:1000 'search_query_latency_median'=0;0:100;0:1000 'search_query_latency_min'=0;0:100;0:1000 'search_query_latency_95'=0;0:100;0:1000 'search_query_latency_99'=0;0:100;0:1000 'search_query_latency_999'=0;0:100;0:1000 'search_index_latency_median'=0;0:100;0:1000 'search_index_latency_min'=0;0:100;0:1000 'search_index_latency_95'=0;0:100;0:1000 'search_index_latency_99'=0;0:100;0:1000 'search_index_latency_999'=0;0:100;0:1000
|
||||
```
|
||||
|
||||
### Erlang Resource Usage Metrics:
|
||||
|
||||
* `sys_process_count`
|
||||
* `memory_processes`
|
||||
* `memory_processes_used`
|
||||
|
||||
#### Sample Command
|
||||
|
||||
```
|
||||
./check_http_json.py -H localhost -P 8098 -p stats -m \
|
||||
"sys_process_count,,0:5000,0:10000" \
|
||||
"memory_processes,,0:50000000,0:100000000" \
|
||||
"memory_processes_used,,0:50000000,0:100000000"
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
OK: Status OK.|'sys_process_count'=1637;0:5000;0:10000 'memory_processes'=46481112;0:50000000;0:100000000 'memory_processes_used'=46476880;0:50000000;0:100000000
|
||||
```
|
||||
|
||||
### General Riak Load / Health Metrics:
|
||||
|
||||
* `node_get_fsm_siblings_mean,_median,_95,_99,_100`
|
||||
* `node_get_fsm_objsize_mean,_median,_95,_99,_100`
|
||||
* `riak_search_vnodeq_mean,_median,_95,_99,_100`
|
||||
* `search_index_fail_one`
|
||||
* `pbc_active`
|
||||
* `pbc_connects`
|
||||
* `read_repairs`
|
||||
* `list_fsm_active`
|
||||
* `node_get_fsm_rejected`
|
||||
* `node_put_fsm_rejected`
|
||||
|
||||
#### Sample Command
|
||||
|
||||
```
|
||||
./check_http_json.py -H localhost -P 8098 -p stats -m \
|
||||
"node_get_fsm_siblings_mean,,0:100,0:1000" \
|
||||
"node_get_fsm_siblings_median,,0:100,0:1000" \
|
||||
"node_get_fsm_siblings_95,,0:100,0:1000" \
|
||||
"node_get_fsm_siblings_99,,0:100,0:1000" \
|
||||
"node_get_fsm_siblings_100,,0:100,0:1000" \
|
||||
"node_get_fsm_objsize_mean,,0:100,0:1000" \
|
||||
"node_get_fsm_objsize_median,,0:100,0:1000" \
|
||||
"node_get_fsm_objsize_95,,0:100,0:1000" \
|
||||
"node_get_fsm_objsize_99,,0:100,0:1000" \
|
||||
"node_get_fsm_objsize_100,,0:100,0:1000" \
|
||||
"riak_search_vnodeq_mean,,0:100,0:1000" \
|
||||
"riak_search_vnodeq_median,,0:100,0:1000" \
|
||||
"riak_search_vnodeq_95,,0:100,0:1000" \
|
||||
"riak_search_vnodeq_99,,0:100,0:1000" \
|
||||
"riak_search_vnodeq_100,,0:100,0:1000" \
|
||||
"search_index_fail_one,,0:100,0:1000" \
|
||||
"pbc_active,,0:100,0:1000" \
|
||||
"pbc_connects,,0:100,0:1000" \
|
||||
"read_repairs,,0:100,0:1000" \
|
||||
"list_fsm_active,,0:100,0:1000" \
|
||||
"node_get_fsm_rejected,,0:100,0:1000" \
|
||||
"node_put_fsm_rejected,,0:100,0:1000"
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
OK: Status OK.|'node_get_fsm_siblings_mean'=0;0:100;0:1000 'node_get_fsm_siblings_median'=0;0:100;0:1000 'node_get_fsm_siblings_95'=0;0:100;0:1000 'node_get_fsm_siblings_99'=0;0:100;0:1000 'node_get_fsm_siblings_100'=0;0:100;0:1000 'node_get_fsm_objsize_mean'=0;0:100;0:1000 'node_get_fsm_objsize_median'=0;0:100;0:1000 'node_get_fsm_objsize_95'=0;0:100;0:1000 'node_get_fsm_objsize_99'=0;0:100;0:1000 'node_get_fsm_objsize_100'=0;0:100;0:1000 'search_index_fail_one'=0;0:100;0:1000 'pbc_active'=0;0:100;0:1000 'pbc_connects'=0;0:100;0:1000 'read_repairs'=0;0:100;0:1000 'list_fsm_active'=0;0:100;0:1000 'node_get_fsm_rejected'=0;0:100;0:1000 'node_put_fsm_rejected'=0;0:100;0:1000
|
||||
```
|
||||
Reference in New Issue
Block a user