Geneos Monitoring Report

Important
Starting with Geneos version 7.7.0, the provided monitoring_report_to_excel.py script will not work with data from Gateways running versions prior to 7.7.0. Similarly, data from Geneos version 7.7.0 onwards cannot be used with the original monitoring report script that was supplied with Geneos versions earlier than 7.7.0.

Introduction Copied

The Geneos Gateway provides a Geneos command that will generate a report of what the Gateway is monitoring. This report is presented in the form of a JSON or YAML document, with example scripts to convert into an Excel workbook.

The report includes the following:

Report information	Specific details
Details of all the probes that are running on the Gateway Note This includes any self-announcing probes that are connected to the Gateway at the time the report is run.	Probe name Probe hostname Probe port OS version of the server the probe resides upon
Details of all the managed entities that are running on the Gateway	Managed entity name All attributes defined on the managed entity
Details of each sampler that are running on the Gateway	Sampler name Sampler type Sampler group Sampler interval of the sampler Active times that are applied to the sampler and affect when it samples Dataviews created by the sampler Any rules that are configured to run on the sampler or any descendant of the sampler (dataview, headline cells, or table cells). These rules are identified by name (rule group, rule name, and rule default name). Any sampler variables that are referenced in the rules will also be listed along with their values. Any rules configured on a sampler or any descendant that have associated KBA links configured on the targets where those rules are applied. This shows a count of how many cells the KBA sets apply to, out of the total number of cells where each rule is evaluated.
Details of all the rules that are available on the Gateway	Rule name Rule group Rule default name Rule logic Rule actions Rule active times
Details of all the active times that are available on the Gateway	Active Time name Active Time description
Details of all configured KBAs and URLs	KBA set name Label name URL

Generating a Gateway report using a Geneos Command Copied

The report can be generated by running the Gateway command /REPORTING:monitoring.

The command is available on the Gateway directory component and the Gateway sampler that is provided as part of the Gateway self-monitoring.
In the Active Console and in the Entity Viewer, the command is called Monitoring. This can be found under the Reporting menu.

Monitoring command

This can be run in all the ways that a Geneos command is available:

From the Active Console
From the Entity Viewer App in the Web Console
Using a Geneos Task
Scheduled using scheduled commands
Via the commands REST API

The results of the command are stored in a file that can be found in the directory specified in the command arguments. The file can either be in JSON (form monitoring_report_<timestamp>.json) or YAML (monitoring_report_<timestamp>.yaml) format.

Note
While YAML is easier for humans to read, JSON is easier to manipulate to generate other data. JSON is ideal for maintaining data types and interacting with APIs or web services, while YAML is useful for creating configuration settings or adding comments in your data because of its user-friendly and readable format.

Gateway Monitoring Report Arguments

Adding Custom Details Copied

Certain information that needs to be captured for the monitoring report can be obtained from:

Headlines and cells in sampler dataviews.
Values in the resolved sampler setup (the sampler configuration after variables have been substituted).

The monitoring report command allows you to provide report configuration in YAML format. This configuration instructs the report to provide detailed information for certain samplers.

Note

You can provide a list of details entries.

Each details entry has a filter that identifies which samplers it applies to, a type that controls the evaluation scope, and the information to extract (columns, headlines, and optionally configExtracts).

The same plugin can appear in more than one details entry. Use different name and type values when you need both sampler-wide setup and per-row or per-dataview data.

# Report YAML that defines the extra detailed information to obtain for the report
report:
  # Indicates whether sampler rules in the report will contain only the rules that have been evaluated.
  showEvaluatedRulesOnly: false
  # Set to true to include per-rule KBA match counts in sampler rule data.
  showRuleKbaMatches: false

  # Array of details - each of which is extracted for each sampler that matches the filters and plugin
  details:
    # Name of the details which is used to wrap the data when placed in the report
    # Details will be placed alongside other information for each sampler that the filters apply to.
  - name: fkm
    # Plugin name of the samplers that these details will be collected for
    plugin: fkm
    # Filters to use to limit which samplers to collect this data for
    filters:
      # Type of the filter (hostname, probe, managedEntity, sampler, samplerType, dataview, attribute)
    - type: attribute
      # Name of the attribute to match against (required only for attribute filters)
      name: environment
      # List of regular expressions to match against the value provided by the type. If any one of the
      # regular expressions match then the details will be extracted
      regex:
      - prod
      - uat

    # Details extraction type. Controls how often columns, headlines, and configExtracts are evaluated:
    # row      - once per dataview row
    # dataview - once per matching dataview
    # sampler  - once per matching sampler (no dataview or row iteration)
    type: row
    # List of columns. Cells in these columns will be extracted from the dataview and placed in the report.
    # This data is only extracted when type is `row`.
    columns:
    - name
    - fileOwner
    - filename
    # List of headlines. These headline cells will be extracted from the dataview and placed in the report.
    headlines:
    - filesMonitored

Details type Copied

Each details entry must set type to one of:

`type`	Evaluation scope	Typical use
`row`	Once per dataview row	Runtime table cells and setup values correlated to a row
`dataview`	Once per matching dataview	Headlines and setup values that apply to a whole dataview
`sampler`	Once per matching sampler	Setup values that apply to the whole sampler (for example, a script path or port list)

Unknown type values are rejected.

Extracting dataview cells and headlines Copied

columns — Table cell values; used when type is row.
headlines — Headline cell values; used when type is row or dataview.

Extracted column and headline values are written to additionalInfo in the report output.

Extracting sampler setup with configExtracts Copied

Use configExtracts when the information you need is in the resolved sampler configuration, but isn’t available or reliable as a dataview cell or headline. Each rule is evaluated against that sampler’s resolved setup XML.

Extracted values are written to additionalInfo under the rule name. Existing reports that omit configExtracts are unchanged.

Each configExtracts entry supports:

Field	Required	Description
`name`	Yes	Output field name in `additionalInfo`.
`xpath`	One of `xpath` or `wpath`	Single-value extraction. A string, or an ordered list of XPath strings. The first expression that yields a usable value is used.
`wpath`	One of `xpath` or `wpath`	Multi-value extraction. A single XPath that can return zero or more values.
`join`	No	Delimiter used only with `wpath`. Defaults to `", "` when omitted.
`mode`	No	`text` (default) or `exists`. `exists` is valid only with `xpath`.

Rules:

A rule must specify exactly one of xpath or wpath. Specifying both, or neither, is invalid and the report request is rejected.
mode: text (default) returns the string value of the matched node. Empty text isn’t treated as a match, so the next xpath in the list is tried. If none of them match, the field is set to not matched.
mode: exists returns "true" if any listed xpath matches at least one node; otherwise, it returns "false". Text content is ignored. Use this mode to detect optional empty elements.
wpath joins all matched values with join (or ", "). If nothing matches, the field is empty.
Sensitive setup values (for example, passwords) are masked in the report output.

XPath expressions are evaluated only within the resolved sampler setup for that sampler. Don’t navigate through <data> wrapper nodes; after variable resolution, those wrappers aren’t a reliable anchor for extraction.

Placeholders Copied

XPath and wpath strings can include placeholders. Values are substituted and escaped for use in XPath string contexts.

Placeholder	Available when	Description
`{{samplerName}}`	`row`, `dataview`, `sampler`	Current sampler name.
`{{dataviewName}}`	`row`, `dataview`	Current dataview name. Empty for `type: sampler`.
`{{rowname}}`	`row`	Current row name. Empty for `type: dataview` or `type: sampler`.
`{{parameter:<name>}}`	All types	Value of the named managed entity attribute.
`{{runtimeParameter:<name>}}`	`row`	Value of the named row runtime parameter (for example, FKM `configIndex`).

If a placeholder doesn’t apply for the current type, or the named value isn’t set, it resolves to an empty string.

Note
For FKM, {{runtimeParameter:configIndex}} is converted from the probe’s 0-based index to a 1-based XPath position, so expressions such as plugin/fkm/files/file[{{runtimeParameter:configIndex}}]/... select the correct configured <file> entry.

Limitations Copied

Row-to-setup linking without configIndex — Reliable correlation of a dataview row to a specific entry in the resolved sampler setup depends on a row runtime parameter such as FKM configIndex. Plugins that don’t expose configIndex (or an equivalent runtime parameter) can’t be reported in a way that reliably links each dataview row to its matching resolved setup entry. You can still extract sampler-wide or dataview-wide setup with type: sampler or type: dataview, or attempt name-based matching with {{rowname}} where the configured value and the runtime row name are identical.
FTM filename wildcards — FTM paths that use wildcards or tokens in the configured filename (for example, <today>) can’t be matched reliably to runtime rows. The resolved setup keeps the wildcard form, but the dataview row name shows the expanded value. As a result, xpath expressions that compare against {{rowname}} don’t extract those entries.
Gateway SQL tables and dataviews — Gateway SQL source tables and output views are configured separately. configExtracts can report table definitions at sampler scope and per-view SQL at dataview scope, but you can’t reliably link source tables to their output dataviews in the report.

Example: TCP-Links sampler and runtime rows Copied

Use a type: sampler entry for setup that applies once per sampler, and a separate type: row entry for runtime table cells:

  - plugin: tcp-links
    name: tcp-links-setup
    type: sampler
    filters:
      - type: managedEntity
        regex:
          - ME
      - type: sampler
        regex:
          - TCPLinks
    configExtracts:
      - name: localPorts
        wpath: plugin/tcp-links/localPorts/port
        join: ", "
      - name: remotePorts
        wpath: plugin/tcp-links/remotePorts/port
        join: " / "
  - plugin: tcp-links
    name: tcp-links-connections
    type: row
    filters:
      - type: managedEntity
        regex:
          - ME
      - type: sampler
        regex:
          - TCPLinks
    columns:
      - name
      - localAddress
      - remoteAddress
      - state
      - type

Example: Gateway SQL tables and per-view SQL Copied

Use two details entries for the same plugin: one sampler-scoped and one dataview-scoped. For the restriction on linking source tables to output dataviews, see Limitations.

  - plugin: gateway-sql
    name: gwsql-tables
    type: sampler
    filters:
      - type: sampler
        regex: [GwSql]
    configExtracts:
      - name: sourceTables
        wpath: plugin/Gateway-sql/tables/dataview/tableName
        join: ", "

  - plugin: gateway-sql
    name: gwsql-views
    type: dataview
    filters:
      - type: sampler
        regex: [GwSql]
    configExtracts:
      - name: viewSql
        xpath: plugin/Gateway-sql/views/view[name="{{dataviewName}}"]/sql

Example: State Tracker path correlated to a row Copied

  - plugin: statetracker
    name: statetracker-job-rows
    type: row
    filters:
      - type: managedEntity
        regex:
          - ME
      - type: sampler
        regex:
          - JobStateTracker
    columns:
      - Job
      - previousState
      - state
      - time
    configExtracts:
      - name: configuredTrackerTemplate
        xpath: plugin/stateTracker/trackerGroup/trackers/tracker[name="SharedJobLog"]/name
      - name: configuredMonitorPath
        xpath: plugin/stateTracker/trackerGroup/trackers/tracker[name="SharedJobLog"]/filename
      - name: configuredTransitionStateNames
        wpath: plugin/stateTracker/trackerGroup/trackers/tracker[name="SharedJobLog"]/transitionStates/*/@name
        join: ", "

The XPath strings in these examples are illustrative. They must match the resolved sampler setup shape for your plugin version and estate.

Extra FKM Columns Copied

For FKM, it is important to be aware of the configuration information associated with each file being monitored. This includes the table cells and headlines extracted from the FKM dataview. The Details sections for the FKM plugin will also provide four additional types of information in a dedicated config object:

fkmActiveTime — the active time associated with the file (if defined)
ignoreKeys — a list of ignore keys for this file
failKeys — a list of fail keys for this file
warningKeys — a list of warning keys for this file

These keys (Critical, Warning, and Ignore) are automatically extracted if the plugin type is fkm. That built-in FKM config output is unchanged.

You can also add configExtracts to an FKM details entry to extract additional setup values into additionalInfo. Use distinct field names so they don’t collide with the built-in config keys.

Note
Built-in FKM config extraction requires Netprobe version 7.1.0 or later.

Generating a Gateway report using a REST request Copied

Enabling a REST Service in the Gateway allows you to directly request a monitoring report. The YAML report configuration can be passed as the POST body of the REST request. A report in JSON format will be returned from the request.

To run the report on an insecure Gateway running on the same machine with no authentication, run the following:

curl -X POST -H 'Content-Type: application/json' http://127.0.0.1:7039/rest/reporting/monitoring

To connect to a Gateway with authentication, refer to user authentication and authorizing an SSO user.

Generating Excel reports Copied

The JSON reports can be converted into an Excel workbook through a python script. This script will convert the JSON reports from multiple Gateways into a single workbook with multiple sheets.

The script (monitoring_report_to_excel.py) is included in the Gateway package at <gateway package>/templates/monitoring_report_to_excel.py. It’s provided as an example and works with reports generated using the default parameters. Values from additionalInfo (including extracted columns, headlines, and configExtract values) are written as additional columns on the details worksheets.

Note
Install the xlsxwriter Python package before you run the script.

You can download a sample JSON report and Excel workbook generated by monitoring_report_to_excel.py.

Previous article Next article

Geneos Monitoring Report

Introduction Copied

Generating a Gateway report using a Geneos Command Copied

Adding Custom Details Copied

Details type Copied

Extracting dataview cells and headlines Copied

Extracting sampler setup with configExtracts Copied

Placeholders Copied

Limitations Copied

Example: TCP-Links sampler and runtime rows Copied

Example: Gateway SQL tables and per-view SQL Copied

Example: State Tracker path correlated to a row Copied

Extra FKM Columns Copied

Generating a Gateway report using a REST request Copied

Generating Excel reports Copied

Was this topic helpful?

Your thoughts...

How can we improve this topic?

Your thoughts...

Thank you for your feedback!