FIX Analyser 2 - The FA2 Netprobe restarts or crashes after modifying the Order Cache Size setting

Restart is due to the Netprobe’s memory protection feature Copied

There are instances where the FIX Analyser 2 Netprobe (aka FA2 Netprobe) keeps on restarting (and sometimes crashes) when the Order Cache Size setting (localCache > orderCacheSize) is set to a value higher than its default. The restarts are triggered by the FA2 Netprobe’s memory protection. The following log entry can be seen in the said Netprobe’s log file:

2021-01-05 10:00:00.600+0000 ERROR: NetProbe Restart Message: IMPORTANT - Contact your support provider; if supported directly by ITRS then mailto:support@itrsgroup.com (1234567) otherwise your usual support contact

There are two scenarios that can trigger the memory protection restart of the FA2 Netprobe:

• Scenario 1: The initial virtual memory doubled.
• Scenario 2: The default maximum virtual memory of 4GB was breached. This has been raised as a JIRA to set the default to unlimited (Look for COL-9409 in the release notes if this gets resolved).

Why is the virtual memory increasing when the Order Cache Size setting is modified Copied

The value that is set in the Order Cache Size setting adds up to the FA2 Netprobe’s total virtual memory consumption. If a large value is used, the FA2 Netprobe’s total virtual memory increases. The computation is:

Total Virtual Memory = Pre-Allocated Virtual Memory + Runtime Virtual Memory

Where:

• If the Order Cache Size setting of all samplers are the same: Pre-Allocated Memory = OrderCacheSize * 1024 * number of samplers
• If the Order Cache Size setting of all samplers are not the same: Pre-Allocated Memory = sum of the OrderCacheSize across all samplers * 1024
• Runtime Virtual Memory is the memory consumed by the FA2 Netprobe to do its normal tasks (e.g. processing files).

The pre-allocated virtual memory of a FA2 Netprobe that has 32 FA2 samplers with 50MB Order Cache Size is 1638400KB (or 1.5625GB):

Pre-Allocated Memory = 50 * 1024 * 32

How to prevent the memory restarts Copied

• Solution 1: Lessen the value of the Order Cache Size setting.
• Solution 2: Split the FA2 samplers into multiple FA2 Netprobes.
• Solution 3: Increase the maximum virtual memory of the FA2 Netprobe by doing the following steps:
  1. Open the FA2 Netprobe’s start script.
  2. Add and export the environment variable MAX_MEM_SIZE= e.g. export MAX_MEM_SIZE=10240 (This sets the maximum virtual memory to 10GB). The upper limit is currently at 16384MB.
  3. Save the changes.
  4. Restart the FA2 Netprobe.