File Keyword Monitor (FKM)

Overview Copied

The universal File Keyword Monitor or FKM plugin provides you a fast and simple method for monitoring multiple files.

This plugin also allows you to:

• Search specific keywords contained within the file.
• Check if the file is updated.
• Check if the file has remained unchanged for a given period of time.

This functionality, in combination with the Gateway rules feature, can quickly inform you of a developing problem. By monitoring a server log file for error messages, you can identify a potential risk or problem on the monitored server via the dataview format.

To view the FKM plugin configuration and settings, see configuration.

Intended audience Copied

This guide is intended for experienced Geneos users who want to configure the FKM plugin to monitor real-time data, and for users who have completed the recommended Geneos course training.

As a user, you should be familiar with Unix and Windows file system structure and permissions, and regular expressions or regex.

Prerequisites Copied

The following requirements must be met prior to the configuration and setup of the FKM plugin:

• You must use the supported versions of Active Console, Gateway, and Netprobe. For guidance, see Supported Geneos versions in 6.x Compatibility Matrix.
• You must have the servers of the Active Console, Gateway, and Netprobe running on the supported operating systems. For guidance, see Operating system support in 6.x Compatibility Matrix.
• You must connect your Active Console to the Gateway where this plugin is configured.
• The Netprobe user must have a read-access permission to the monitored file.

Introduction Copied

FKM also allows quick viewing of any text-based monitored files so users can see the content directly from Active Console, enabling a fast response to issues. Viewing functionality can also be restricted to specific users or user groups for sensitive files.

File Keyword Monitor has two main modes:

• Single-trigger — shows one trigger row per key, per file. This means that if the same key appears twice or more in a file, then only the details for the latest-detected line matching that key will be shown.
• Multiple-trigger — shows a new trigger row for each detected key. Detected keys for each file are then indexed by increasing number starting at 0. Later numbers indicate later detected keys.

Stream-based inputs such as TibRV messages can be monitored in conjunction with the appropriate plugins, as well as Windows Event Logs. The FKM plugin operates by reading multiple files that appear as a row in the published dataview. Each file can be configured with a set of fail keys and warning keys, and has a variety of scanning types.

Another feature of FKM is to dynamically ignore known problems on a single machine without reconfiguring the plugin. Ignore files provide a mechanism for configuring ignore keys for an FKM file externally to the gateway setup.

An ignore key can be used to disregard the known messages in a server log that may imply a possible error, but these messages are plain scripts or system-defined. Ignore keys should be defined before the fail or warning keys in a table, because keys in a key table are checked against a file line in order of definition

Create a sampler in GSE Copied

Ensure that you have read and can follow the prerequisites prior to installation and setup of this plug-in.

To setup and configure the FKM plug-in, follow these steps:

1. Open Active Console.

2. Double-click the running Gateway to open Gateway Setup Editor.

3. In Gateway Setup Editor, click Samplers in the Navigation panel.

4. Click New Sampler.

5. In the Samplers section, select FKM on the Plugin menu.

6. Click Save current document to apply changes.

   

Here are the fields that you can configure on the FKM tab:

Monitor a source file in FKM Copied

Now that you have created an FKM sampler, you can start configuring its basic settings to monitor a file. Creating a new sampler for each monitored log file is not required. One managed entity group can hold multiple samplers, and one sampler can monitor multiple logs files. Once completed, the newly created FKM plug-in displays in the Samplers section.

Note

The Netprobe user ID must have a read-access to read the source file.

1. Open the Basic tab of the FKM sampler in Gateway Setup Editor.

2. Click Add new to start a monitoring a file.

3. Enter the path location of the file you want to monitor in the Source > filename field. For example, location/etc/log.txt.

4. Select the file format that matches the file to be monitored on the Content type menu.

5. Click Save current document to apply changes.

6. Check the Active Console dataview if it displays the configuration details of the FKM plug-in.

   Caution: The FKM plug-in always opens and scans a file for reading, and scans each incoming line. The FKM plug-in performs these tasks even if you choose not to define search keys. Therefore, you should not use the FKM plug-in to detect core files, as it reads core files as if they were text files. Core file detection is better served by a toolkit.

If the FKM dataview does not show any results, these are the possible errors:

• File does not exist.
• File is inaccessible or it has no read-access from the Netprobe. Ensure that the Netprobe has the necessary permissions to access the file and all the directories leading to the file.

When performing wildcard matches on a Unix server and there are sufficient file permissions to read and access the file, then Active Console displays OK in the status column:

On Windows, ensure that the Netprobe has the necessary permissions to both the directories and the files. If there are sufficient file permissions, then the Active Console displays OK in the status column:

For more information, see permissions in File Keyword Monitor Plug-in Technical Reference.

If there are no sufficient file permissions, then you may encounter these error messages:

• "NOT_FOUND: Permission Denied — the Netprobe does not have permissions to traverse higher level directories. This can also occur if the wildcard pattern can match multiple directories, some of which are protected, and no matching paths leading to the file are found.
• "NOT_FOUND: No such file or directory" — all directories are searchable, but no file matches.

Usually, the Netprobe runs as a Windows Service and this process has no access to mapped drives. Therefore, FKM can only access local files.

To view the other options that you can configure, see File Keyword Monitor Plug-in Technical Reference.

Configure FKM plugin Copied

This sections provides other use cases and scenarios that you can configure in the FKM plug-in.

Monitor a log file for a specific error message Copied

This functionality allows the log files to quickly identify messages that are relevant to the user. It also helps the user to interpret the output in the dataview when the log file is updated.

1. In Gateway Setup Editor, open the FKM sampler you have created.

2. On the Basic tab, enter the filename you want to be monitored in the Source > filename field. The value specifies the location of the file. The Netprobe working directory evaluates the relative paths.

   Note

   If a filename contains wildcard characters, then FKM automatically checks for creation of newer files matching the wildcard pattern. When a new file is detected, FKM switches to monitor the newer file if the current file has been scanned to the end.

3. Click Tables.

4. Click Add new on the Tables window. When the string is found, set the Severity option depending on the business requirement.

5. Select Fail on the Severity menu.

6. Select data on the Key table menu.

   

7. In the Tables group, click Data. This opens the Data window.

8. Click Data > Keys. This opens the Key window.

9. Select Match on the Set key menu. A match set key specifies the text that must appear in the log file to causing a match.

10. Enter the specific error message you want to monitor in a file in the Set key > Search string field.

    

11. Select Basic on the Rules menu. Basic matching searches the file line of the Search string using case insensitive comparison.

12. Go back to the FKM sampler view in the Gateway Setup Editor.

13. Click Save to apply changes.

14. Open the Active Console dataview to view the FKM sampler you have created.

15. If you do not have a log file that is being regularly updated, use simulation to update your monitored file. Check if it can monitor the search string keyword with the expected status:

16. Return to FKM dataview in Active Console where the sampling is and see if it picks up the connection error message. Referring to the status column, the cell value has changed from OK to FAIL because it matches the search string connection error and the Severity ruling which was set to “Fail”.

17. Scroll to the right of the dataview to view the triggerDetails column. This displays the line containing the matched string connection error:

Set FKM in multiple trigger mode Copied

This setting allows you to view all lines that match the key in the dataview. To configure the FKM plug-in, follow these steps:

1. In Gateway Setup Editor, open the FKM sampler you have created.
2. Click the Advanced tab.
3. Select Multiple on the Trigger mode menu. In multiple trigger mode, FKM shows a new trigger row for each detected key. Detected keys for each file are then indexed by increasing number starting at 0. Later numbers indicate later detected key.

7. Click Save current document to apply changes.
8. Return to the server to update the log file.
9. Verify that the dataview has recorded the update with a trigger row.

The advantage of setting the trigger mode to multiple is the previous record remains in the dataview, and is not replaced by a new set of data.

Display more than one trigger row per key Copied

By default, the FKM plug-in only displays the most recently detected line that matches the key. If you want to see all the lines matching the key, you must use the multiple trigger mode.

1. In Gateway Setup Editor, open the FKM sampler you have created.

2. Click the Advanced tab.

3. Select Single on the Trigger mode menu. In single trigger mode, FKM only shows one trigger row per key and per file. This means that if the same key appears twice (or more) in a file, then the dataview only displays the details for the latest detected line matching that key.

   Note

   If you do not have a log file that is being regularly updated, use simulation to update your monitored file. Create a file that you can concatenate into the log file to be monitored.

4. Click Save current document to apply the changes.

5. Cat the file to the end of the log file.

   

6. Return to the dataview to check that the addition to the log file has been detected and registered.

7. If no, return to the server and use the cat command to correct the error in the file.

   

8. Check that the existing trigger row has been replaced by the latest update to the log file.

The lastModificationTime values have changed and the previous row has been deleted.

Clear a trigger row Copied

This allows you to clear a trigger row. This is useful if you have set a key match with a string such as “Connection Error”, but you know that the problem can be quickly fixed. To clear the trigger, you can set a key to match a second string, for example “Reconnected”. Upon matching the second string, this clears the trigger row.

When FKM detects the first string connection error, it flags this in the dataview by creating a trigger row. When it detects the second string reconnected, it automatically clears the trigger row.

1. In Gateway Setup Editor, open the FKM sampler you have created.

2. On the Basic tab, click Tables.

3. Click Table > Data. This opens the Data window.

4. Click Data > Key. This opens the Key window.

5. In the Clear key group, type “Reconnected” in the Search string field. When FKM spots “Reconnected” in the log file, this automatically clears the key from the dataview.

   

6. Select Basic on the Clear key > Rules menu. There is no regular expression or regex needed on this setup.

7. Close the screens until you return on the FKM Basic tab.

8. Click Save current document to apply changes.

9. Return to the dataview and verify that the trigger has been cleared.

10. Wait until the dataview reaches the sampling time, then refresh the data. Once completed, this automatically clears a trigger when a problem has been fixed

Display a customised message in the dataview Copied

This allows you to display a relevant message in the triggerDetails column of the FKM dataview. This is used to specify any error messages encountered.

1. In Gateway Setup Editor, open the FKM sampler you have created

2. On the Basic tab, click Tables.

3. Click Table > Data. This opens the Data window.

4. Click Data > Key. This opens the Key window. The Key button brings up the previous configuration details you have setup for Search string and Rules fields.

5. In the Message field, enter the specific message you want to display in the dataview. Ensure the display message is clear and can be easily understood by all users.

   

6. Close the screens until you return on the FKM Basic tab.

7. Click Save current document to apply changes.

8. If you have a running setup, check that the message has been displayed in the status column of the file row and in the triggerDetails column of the trigger row.

The Status field shows the specified error message:

The triggerDetails field shows the specified error message.

Extract a fragment of a line in a log file Copied

This allows you to extract part of a line in a log file. This can be useful when many similar messages are sent to a log file, but most of them are of no interest. The Geneos Extractor feature allows you to search a specific range of characters, for example all numbers between 1020 and 3780, excluding all the numbers before and after the range:

Note

The prerequisite for this configuration setup is to set the trigger mode to multiple.

Set up an extractor Copied

Note

You can create a dummy log file and files that you can use to simulate an application writing to a log file.

1. In Gateway Setup Editor, open the FKM sampler you have created.

2. On the Basic tab, click Tables.

3. Click Table > Data. This opens the Data window.

4. Click Data > Key. This opens the Key window.

5. Enter the search string value in the Search string field. The sample regular expression in the screenshot will look for digits from 0-9:

6. Select Regexp on the Rules menu.

7. Click Add new in the Extractors section.

8. Enter the name of the extractor in the Name field.

   Note

   • Extractors can have the same names if they are under different keys.
   • The name of the extractor must not be the same as the name of an FKM column.

9. Enter a valid regular expression in the Regex field. The regular expression must have one capture group only.

   

10. Close the screen until you return on the FKM Basic tab.

11. Click Save current document to apply changes.

12. Check the FKM dataview in the Active Console to verify that the column has been created from the Extractor section. In this case, Error Code has been added to the FKM dataview.

    

Create a rule Copied

1. In Gateway Setup Editor, click the Rules section.

2. Click Create a new item > New Rule.

3. Enter the name of the rule in the Name field.

4. Enter the rule expression in the Block field. This is where the rule codes are created. It is evaluated each time any relevant data changes.

5. Right-click the Block field to view the most common keywords and functions.

   

   In this example, the set rule is created to test the value against the two numbers (2859 and 1182). When the program finds one of this numbers, it runs the clearTrigger action. This action automatically removes the identified values from the dataview.

6. Once the rule is set, identify the target path.

7. Return to the FKM dataview in Active Console.

8. Right-click the cell of the column where the rule is to be implemented.

9. Click Copy > Path. An example of path directory:

   /geneos/gateway[(@name="GATEWAY_68944")]/directory/probe[(@name="New Probe")]/managedEntity[(@name="Exercises")]/sampler[(@name="fkmtest")][(@type="")]/dataview[(@name="fkmtest")]/rows/row[(@name="/sbox/home/useralias/etc/fkm_log.txt#fail00000")]/cell[(@column="lastModificationTime")]
   

10. Return to the Rule section in GSE.

11. Paste the path of the cell in the Targets field.

12. Click Edit to open the path editor.

13. Click the Evaluate Path button to validate the correctness of the identified path.

    Note

    If at least one of the elements have turned into red field, there is no match found in the validated path. Ensure the source path is accurately correct.

14. Click Save current document to apply changes.

Execute an action Copied

1. In Gateway Setup Editor, click the Actions section.

2. Click Create a new item > New Action.

3. Enter the name of the action in the Name field to create an internal command.

4. Select Internal command on the Options menu.

5. In the Internal command group, select FKM:clearTrigger on the Name menu.

   

6. Click Save current document to apply changes.

At this stage, the search string, key dialog, and extractor are looking for specific numbers, and a rule to check those numbers has been configured. If it matches one of the identified numbers, then it triggers the action.

For example, if the number does not match the value set in the rule, then it displays in the Error Code row. The value 123 appears in the Error Code row:

If the number matches any of the value set in the rule, it displays in the FKM dataview row. However, due to the action made which is clearTrigger, this automatically disappears from the dataview rowafter a few seconds.

Use wildcard when you enter filenames Copied

This allows you to use wildcard in filenames configuration. This can be useful in a situation where you have several filenames with the same extension, for example .log, and you do not want to enter each filename individually into the sampler.

1. In Gateway Setup Editor, open the FKM sampler you have created.
2. On the Basic tab, scroll to the right to click Add new. The Add new button adds a new Source field in the Files section.
3. Enter the full path name in the newly added Source filename. Use the wildcard by adding an asterisk (*) before the log extension name in the filename.
4. Click Tables > Add new > Data.
5. Click Data > Key. The Key button brings up the previous configuration details you have setup for Search string and Rules fields.
6. Enter the search string value in the Search string field.
7. Select Basic on the Rules menu.
8. Close the screens until you return on the FKM Basic tab.
9. Open the Advanced tab.
10. On the Advanced tab, scroll down until you see the fields related to wildcard.
11. Select the Wildcard monitor all matches check box.
    • If this check box is not activated, the FKM dataview only displays the latest file to be changed.
    • If this is selected, it displays all the files matching the asterisk (*) .config or the log extension name you used.
12. Click Save current document to apply changes.
13. Check the FKM dataview in Active Console to view all the files that match the wildcard.

View file in FKM dataview Copied

The View File command option allows you to remotely view the file being monitored by FKM.

There are two file viewing modes:

• Snapshot view — this only displays the selected parts of the file at the time when the query is made
• Continuous view — displays the same as the snapshot, but is continue to update the file view window when additional lines are added to the file.

1. In Active Console, open the FKM sampler.
2. Right-click the name of the file you want to view.
3. Click View File. The View File is an example of a command that are specific to some plugins or components. Not all cells in the dataview has this functionality.
4. The View File window opens.

The View File dialog allows users to specify how much of the file to view is relative to the end of the file. The maximum size of data from the file that can be displayed is 1 megabyte (1024 kilobytes). This is to prevent performance slowdown caused by the handling of large files that are sent over to the view.

Accept files in FKM dataview Copied

The Accept commands let you acknowledge a file and clear its trigger.

1. In the Active Console State Tree, click to expand the relevant Managed entity.
2. Under the Managed entity, click to expand the relevant probe.
3. Double-click the relevant FKM sampler. The dataview for the sampler appears.
4. In the FKM dataview, right-click the file with the trigger and click the accept option you wish to use. For guidance, see the options below.
5. If you are accepting multiple files, select your desired Accept and Location options in the Accept multiple files window, then click OK.

Success

The trigger is cleared and the file status of the accepted files return to normal.

• Accept this File — accepts triggers of the selected file.
• Accept files… — opens the Accept multiple files window.

When you select Accept files…, you have the following options:

Accept options:

• this file — accepts triggers of the selected file in the locations specified.
• all files — accepts triggers of all files listed in the dataview in the locations specified.

Location options:

• this managedEntity (host) — accepts triggers for files under the Managed entity.
• this sampler — accepts triggers for files under the sampler.
• this probe — accepts triggers for files for all FKM samplers under the Netprobe.
• this gateway (all hosts) — accepts triggers for files for all FKM samplers under the Gateway.

Change FKM dataview columns Copied

The column setting allows you to configure the list of columns to be displayed by FKM in the dataview. You may want to add or remove any existing columns, depending on the needed information to be displayed in the dataview.

1. In Gateway Setup Editor, open the FKM sampler you have created.
2. Open the Advanced tab.
3. Click Columns > Add new.
4. On the Column menu, select the column name you want to be displayed in the dataview.
5. Click Save current document to apply changes.

Limit the numbers of displays triggers Copied

Use the maxConditionPerKey parameter to control the trigger rows or conditions that are recorded per file. This is used primarily in multiple trigger mode and single grouped message mode to limit the number of triggers to be created.

In this use case, set the trigger mode to multiple and use the maxConditionPerKey to limit the number of rows which by default displays more than 1,000 rows or lines of triggers.

1. In Gateway Setup Editor, open the FKM sampler you have created.

2. Open the Basic tab and click Tables.

3. In the Table section, complete the necessary fields:

   • Select Fail on the Severity menu.
   • Select data on the Key table

4. Click Tables > Data > Key. This opens the Key window.

5. Enter the keyword in the Search string field. In this example, the search string is “test”.

6. Go back to the FKM in GSE.

7. Open the Advanced tab.

8. Select Multiple on the Trigger mode menu.

9. Scroll down until you see the maxConditionPerKey parameter.

10. Enter the limit number to be displayed in the Max conditions per key field.

    Note

    If a new trigger arrives after this limit has been reached, the older trigger for the key will be removed and replaced with the new trigger.

11. Click Save current document to apply changes.

Use date generation function when using dynamic files option Copied

The dynamicFiles file source type configures FKM to match groups of files based on the configured path, pattern, and optionally the alias.

To use the date generation function when the dynamicFiles option is selected, follow these steps:

1. In the Gateway Setup Editor, open the FKM sampler you have created.

2. Go to Basic tab > Files > Source, and select dynamicFiles.

3. Click Dynamic files. This opens the Dynamic files window.

4. In the Dynamic files > Path field, enter the path that you want to monitor. For example, /opt/geneos/gateway/<today %Y>/<today %m>/<today %d>/*.txt.

5. In the Pattern > Data > Regex field, enter the expression based on the files you want to match. In this example, enter ^.*.txt.

6. Click Close.

   

7. On the Advanced tab, tick the Use PCRE Perl Compatible Regex box.

8. Click Save current document to apply changes.

For more information, see files > file > source > dynamicFiles in FKM Plug-In: Technical Reference.

Create a rule to ensure the last modification time is from today Copied

You can create a rule to compare the last modification time with the current time. With this rule, you can check if the lastModificationTime is from today.

To create a rule:

1. In the Gateway Setup Editor, click Rules > New Rule.

2. Create a rule in the Block field. For example:

   set $(timestamp) parseDate("%a %b %d %H:%M:%S %Y", value)
   if $(timestamp) < startOfDay() then
   severity warning
   else
   severity ok
   endif
   

3. Click Save current document to apply changes.

If the last time stamp is recorded before midnight of the previous day, then the system displays a warning. Otherwise, the severity level displayed is OK.

Further reading Copied

The File Keyword Monitor plugin can be configured to run on any Netprobe host. Configuration should consist of at least one file to be monitored, which will provide basic statistics about the file.

To know more about FKM settings, see configuration.