How to fine tune the performance and resource consumption for FKM plugin?
The FKM plugin is generally used to monitor log files for error patterns. Users may have a variety of system and application log files that can grow rapidly and with complex content, such as application transaction logs and FIX protocol. The Netprobe might use additional resources depending on the environment and configuration. This article provides the common tips to fine tune the configuration.
Identify any problematic sampler or log file Copied
Users can have multiple log files being monitored on each server. The application behavior may also change depending on usage and time of the day.
The FKM plugin reports an updateRate column which is an indicator how busy a monitored log file. Users can configure gateway rules to flag exceptions as needed. For more information, see documentation about excessiveUpdatesTimeout in relation to FKM stopped on this file - excessive update rate detected.
If a user has difficulty determining which sampler or log is causing problems, the SAMPLING debug option can be enabled.
- In the Gateway Setup Editor, select Probes > (Probe Name) > Debug tab.
- Select the Add New button.
- Add SAMPLING with Setting
*as in the screen capture.
The netprobe log will contain the start and end times when a sampler is executed. This will help identify if a certain sampler is taking a long time to finish. If a monitored log is not located on the local drive (on a remote NFS or NAS storage), there may be considerable delays while accessing the file. Users should take note of the potential performance impacts.
Review FKM keys Copied
If the FKM keys are configured to use regular expressions (regex), the performance can have a big difference depending on the regex pattern configured. In particular, look out for the generic catch all pattern .\ and avoid putting this near the beginning, which can cause unnecessary backtracking and therefore poor performance. Many articles on this topic are on the internet, which can be found with the keyword “regular expression backtracking”.
Arrange the order of FKM keys from specific to generic patterns. You can check the article Does the order of FKM Keys impact performance? for further information.
POSIX vs PCRE regex option Copied
The FKM plugin can select to use POSIX or Perl Compatible Regular Expressions (PCRE) syntax for regular expressions. For more information, see documentation about Notable differences between POSIX and PCRE.
If you do not use regex syntax specific to POSIX or PCRE, you may try toggling the parameter and see if there are any performance improvements. The checkbox option is located at FKM plugin’s Advanced tab. Use PCRE Perl Compatible Regex.
Note
The FKM plugin had POSIX as its default in older versions of Netprobe. But the default has switched to PCRE since Geneos version 5.4.x (COL-7588).
For more information, see Will there will be any performance issue if an FKM sampler look for a pattern on a large txt/log file? and documentaion about File Keyword Monitor (FKM).