File Keyword Monitor (FKM) PlugIn User Guide

Overview

The universal File Keyword Monitor or FKM plugin provides you a fast and simple method for monitoring multiple files. This plugin also allows you to:

• Search specific keywords contained within the file.
• Check if the file is updated.
• Check if the file has remained unchanged for a given period of time.

This functionality, in combination with the Gateway rules feature, can quickly inform you of a developing problem. By monitoring a server log file for error messages, you can identify a potential risk or problem on the monitored server via the dataview format.

Intended audience

This guide is intended for experienced Geneos users who want to configure the FKM plugin to monitor real-time data, and for users who have completed the recommended Geneos course training.

As a user, you should be familiar with Unix and Windows file system structure and permissions, and regular expressions or regex.

Prerequisites

The following requirements must be met prior to the configuration and setup of the FKM plugin:

• You must use the supported versions of Active Console, Gateway, and Netprobe. For guidance, see Supported Geneos versions in 5.x Compatibility Matrix.
• You must have the servers of the Active Console, Gateway, and Netprobe running on the supported operating systems. For guidance, see Operating system support in 5.x Compatibility Matrix.
• You must connect your Active Console to the Gateway where this plugin is configured.
• The Netprobe user must have a read-access permission to the monitored file.

Introduction

FKM also allows quick viewing of any text-based monitored files so users can see the content directly from Active Console, enabling a fast response to issues. Viewing functionality can also be restricted to specific users or user groups for sensitive files.

File Keyword Monitor has two main modes:

• Single-trigger — shows one trigger row per key, per file. This means that if the same key appears twice or more in a file, then only the details for the latest-detected line matching that key will be shown.
• Multiple-trigger — shows a new trigger row for each detected key. Detected keys for each file are then indexed by increasing number starting at 0. Later numbers indicate later detected keys.

Stream-based inputs such as TibRV messages can be monitored in conjunction with the appropriate plugins, as well as Windows Event Logs. The FKM plugin operates by reading multiple files that appear as a row in the published dataview. Each file can be configured with a set of fail keys and warning keys, and has a variety of scanning types.

Another feature of FKM is to dynamically ignore known problems on a single machine without reconfiguring the plugin. Ignore files provide a mechanism for configuring ignore keys for an FKM file externally to the gateway setup.

An ignore key can be used to disregard the known messages in a server log that may imply a possible error, but these messages are plain scripts or system-defined. Ignore keys should be defined before the fail or warning keys in a table, because keys in a key table are checked against a file line in order of definition

Create a sampler in GSE

Ensure that you have read and can follow the prerequisites prior to installation and setup of this plug-in.

To setup and configure the FKM plug-in, follow these steps:

1. Open Active Console.
2. Double-click the running Gateway to open Gateway Setup Editor.
3. In Gateway Setup Editor, click Samplers in the Navigation panel.
4. Click New Sampler.
5. In the Samplers section, select FKM on the Plugin menu.
6. Click Save current document to apply changes.

Here are the fields that you can configure on the FKM tab:

Field	Description
Name	Specifies the sampler name you created in the managed entity. Mandatory: Yes The name of the sampler must be unique among all other samplers. The first character of your sampler name cannot be a special character.
Group	Specifies a group name. Mandatory: No
Description	Specifies a description for your plug-in. Mandatory: No
Sample interval	Controls the period between two samples taken by the sampler, which is measured in seconds. The value should be set to a non-negative integer value. A value of 0 indicates that regular sampling should be disabled. Mandatory: No Default: 20 Unit: Seconds
Basic tab	See File Keyword Monitor Plug-in Technical Reference.
Advanced tab	See File Keyword Monitor Plug-in Technical Reference.

Monitor a source file in FKM

Now that you have created an FKM sampler, you can start configuring its basic settings to monitor a file. Creating a new sampler for each monitored log file is not required. One managed entity group can hold multiple samplers, and one sampler can monitor multiple logs files. Once completed, the newly created FKM plug-in displays in the Samplers section.

Note: The Netprobe user ID must have a read-access to read the source file.

1. Open the Basic tab of the FKM sampler in Gateway Setup Editor.
2. Click Add new to start a monitoring a file.
3. Enter the path location of the file you want to monitor in the Source > filename field. For example, location/etc/log.txt.
4. Select the file format that matches the file to be monitored on the Content type menu.
5. Click Save current document to apply changes.
6. Check the Active Console dataview if it displays the configuration details of the FKM plug-in.

Caution: The FKM plug-in always opens and scans a file for reading, and scans each incoming line. The FKM plug-in performs these tasks even if you choose not to define search keys. Therefore, you should not use the FKM plug-in to detect core files, as it reads core files as if they were text files. Core file detection is better served by a toolkit.

If the FKM dataview does not show any results, these are the possible errors:

• File does not exist.
• File is inaccessible or it has no read-access from the Netprobe. Ensure that the Netprobe has the necessary permissions to access the file and all the directories leading to the file.

When performing wildcard matches on a Unix server and there are sufficient file permissions to read and access the file, then Active Console displays OK in the status column:

On Windows, ensure that the Netprobe has the necessary permissions to both the directories and the files. If there are sufficient file permissions, then the Active Console displays OK in the status column:

For more information, see permissions in File Keyword Monitor Plug-in Technical Reference.

If there are no sufficient file permissions, then you may encounter these error messages:

• "NOT_FOUND: Permission Denied — the Netprobe does not have permissions to traverse higher level directories. This can also occur if the wildcard pattern can match multiple directories, some of which are protected, and no matching paths leading to the file are found.
• "NOT_FOUND: No such file or directory" — all directories are searchable, but no file matches.

Usually, the Netprobe runs as a Windows Service and this process has no access to mapped drives. Therefore, FKM can only access local files.

To view the other options that you can configure, see File Keyword Monitor Plug-in Technical Reference.

Configure FKM plugin

This sections provides other use cases and scenarios that you can configure in the FKM plug-in.

Use wildcard when you enter filenames

This allows you to use wildcard in filenames configuration. This can be useful in a situation where you have several filenames with the same extension, for example .log, and you do not want to enter each filename individually into the sampler.

1. In Gateway Setup Editor, open the FKM sampler you have created.
2. On the Basic tab, scroll to the right to click Add new. The Add new button adds a new Source field in the Files section.
3. Enter the full path name in the newly added Source filename. Use the wildcard by adding an asterisk (*) before the log extension name in the filename.
4. Click Tables > Add new > Data.
5. Click Data > Key. The Key button brings up the previous configuration details you have setup for Search string and Rules fields.
6. Enter the search string value in the Search string field.
7. Select Basic on the Rules menu.
8. Close the screens until you return on the FKM Basic tab.
9. Open the Advanced tab.
10. On the Advanced tab, scroll down until you see the fields related to wildcard.
11. Select the Wildcard monitor all matches check box.
• If this check box is not activated, the FKM dataview only displays the latest file to be changed.
• If this is selected, it displays all the files matching the asterisk (*) .config or the log extension name you used.
12. Click Save current documentto apply changes.
13. Check the FKM dataview in Active Console to view all the files that match the wildcard.

View file in FKM dataview

The View File command option allows you to remotely view the file being monitored by FKM.

There are two file viewing modes:

• Snapshot view — this only displays the selected parts of the file at the time when the query is made
• Continuous view — displays the same as the snapshot, but is continue to update the file view window when additional lines are added to the file.

1. In Active Console, open the FKM sampler.
2. Right-click the name of the file you want to view.
3. Click View File. The View File is an example of a command that are specific to some plugins or components. Not all cells in the dataview has this functionality.
4. The View File window opens.

The View File dialog allows users to specify how much of the file to view is relative to the end of the file. The maximum size of data from the file that can be displayed is 1 megabyte (1024 kilobytes). This is to prevent performance slowdown caused by the handling of large files that are sent over to the view.

Accept files in FKM dataview

The Accept commands let you acknowledge a file and clear its trigger.

1. In the Active Console State Tree, click to expand the relevant Managed entity.
2. Under the Managed entity, click to expand the relevant probe.
3. Double-click the relevant FKM sampler. The dataview for the sampler appears.
4. In the FKM dataview, right-click the file with the trigger and click the accept option you wish to use. For guidance, see the options below.
5. If you are accepting multiple files, select your desired Accept and Location options in the Accept multiple files window, then click OK.

Success: The trigger is cleared and the file status of the accepted files return to normal.

• Accept this File — accepts triggers of the selected file.
• Accept files... — opens the Accept multiple files window.

When you select Accept files..., you have the following options:

Accept options:

• this file — accepts triggers of the selected file in the locations specified.
• all files — accepts triggers of all files listed in the dataview in the locations specified.

Location options:

• this managedEntity (host) — accepts triggers for files under the Managed entity.
• this sampler — accepts triggers for files under the sampler.
• this probe — accepts triggers for files for all FKM samplers under the Netprobe.
• this gateway (all hosts) — accepts triggers for files for all FKM samplers under the Gateway.

Change FKM dataview columns

The column setting allows you to configure the list of columns to be displayed by FKM in the dataview. You may want to add or remove any existing columns, depending on the needed information to be displayed in the dataview.

1. In Gateway Setup Editor, open the FKM sampler you have created.
2. Open the Advanced tab.
3. Click Columns > Add new.
4. On the Column menu, select the column name you want to be displayed in the dataview.
5. Click Save current document to apply changes.

Limit the numbers of displays triggers

Use the maxConditionPerKey parameter to control the trigger rows or conditions that are recorded per file. This is used primarily in multiple trigger mode and single grouped message mode to limit the number of triggers to be created.

In this use case, set the trigger mode to multiple and use the maxConditionPerKey to limit the number of rows which by default displays more than 1,000 rows or lines of triggers.

1. In Gateway Setup Editor, open the FKM sampler you have created.
2. Open the Basic tab and click Tables.
3. In the Table section, complete the necessary fields:
• Select Fail on the Severity menu.
• Select data on the Key table
4. Click Tables > Data > Key. This opens the Key window.
5. Enter the keyword in the Search string field. In this example, the search string is "test".
6. Go back to the FKM in GSE.
7. Open the Advanced tab.
8. Select Multiple on the Trigger mode menu.
9. Scroll down until you see the maxConditionPerKey parameter.
10. Enter the limit number to be displayed in the Max conditions per key field.

Note: If a new trigger arrives after this limit has been reached, the older trigger for the key will be removed and replaced with the new trigger.

11. Click Save current document to apply changes.

Use date generation function when using dynamic files option

The dynamicFiles file source type configures FKM to match groups of files based on the configured path, pattern, and optionally the alias.

To use the date generation function when the dynamicFiles option is selected, follow these steps:

1. In the Gateway Setup Editor, open the FKM sampler you have created.
2. Go to Basic tab > Files > Source, and select dynamicFiles.
3. Click Dynamic files. This opens the Dynamic files window.
4. In the Dynamic files > Path field, enter the path that you want to monitor. For example, /opt/geneos/gateway/<today %Y>/<today %m>/<today %d>/*.txt.
5. In the Pattern > Data > Regex field, enter the expression based on the files you want to match. In this example, enter ^.*.txt.
6. Click Close.
7. On the Advanced tab, tick the Use PCRE Perl Compatible Regex box.
8. Click Save current document to apply changes.

For more information, see files > file > source > dynamicFiles in File Keyword Monitor Plug-in Technical Reference.

Create a rule to ensure the last modification time is from today

You can create a rule to compare the last modification time with the current time. With this rule, you can check if the lastModificationTime is from today.

To create a rule:

1. In the Gateway Setup Editor, click Rules > New Rule.
2. Create a rule in the Block field. For example:

set $(timestamp) parseDate("%a %b %d %H:%M:%S %Y", value)
if $(timestamp) < startOfDay() then
severity warning
else
severity ok
endif

3. Click Save current document to apply changes.

If the last time stamp is recorded before midnight of the previous day, then the system displays a warning. Otherwise, the severity level displayed is OK.