FIX Analyser 2 Netprobe User Guide

Installing the database

Please refer to the MySQL installation instructions for your platform available using the link below: http://dev.mysql.com/doc/refman/5.1/en/installing.html

FIX Analyser 2 requires a MySQL event scheduler feature. This scheduler is used to automate data housekeeping tasks allowing for low-maintenance operation of the message database.

To view the list of supported database, see Database support in 5.x Compatibility Matrix.

Persisting the event scheduler

The FIX Analyser 2 schema install script will enable the event scheduler; however, to ensure the setting persists across restarts of the MySQL server process, it must be set to ON via a command-line option or in the options file. For example, on a Linux system containing a standard MySQL package installation add the line event-scheduler=ON to the /etc/my.cnf configuration file.

Please see the following link for full details: http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html#sysvar_event_scheduler

Attended scripted installation

A schema installation / upgrade script is provided in the resources/db-schema folder of the FIX Analyser 2 package. The script should be run from within this folder and will require database administrator (DBA) access to the database. If remote DBA access has been disabled, then the script will need to be executed on the database server host itself (or remote access granted).

To execute the script and create a new FIX Analyser 2 installation, run the following command:

cd resources/db-schema
./fix_analyzer_schema_install.sh -n

The script will prompt for the following information during the installation process:

• Database administrator (DBA) username
  • This account will perform administrative functions on the database
• DBA password
• Hostname for the MySQL database server
• Port for the MySQL database server
• Name of the FIX Analyser 2 database to create
  • This database should not already exist on the server
• FIX Analyser user account name to create
  • This account will be given permissions to write to the new database
• FIX Analyser user account password
• Time to run the daily data housekeeping
  • Default time is 00:15
• Number of days to retain message data
  • Default is 3 days of data

Unattended scripted installation

The installation script described can also be run in unattended (non-interactive) mode.

To do this, the requisite information the script requires must be provided as environment variables. The table below describes the environment variables the script expects when installing a new database.

Manual installation

If you are unable to or do not wish to install the database schema using the installation script, you can install the schema manually using the following steps. The SQL statements below should be executed as a database administrator (DBA) account on the MySQL database server.

1. Connect to the MySQL database using the MySQL command-line client. See the reference (http://dev.mysql.com/doc/refman/5.1/en/mysql.html) for full details.

mysql -u dba_username -p dba_password [ -h db_host -P db_port ]

2. Enable the MySQL event scheduler.

SET GLOBAL event_scheduler = ON;
SHOW PROCESSLIST;

The output of the SHOW PROCESSLIST command should return a row with the text event_scheduler if it was successful.

3. Create a new database for use by the FIX Analyser 2 plug-in (e.g. FIX_DB).

CREATE DATABASE FIX_DB;

4. Create a user for FIX Analyser 2 to connect to the MySQL server with. We allow this fix_db user to connect from any hostname with the password fix_pw.

CREATE USER fix_db@'%' IDENTIFIED BY 'fix_pw';

5. Grant the user appropriate permissions to access the new database.

GRANT event,execute,select,insert,update,delete on FIX_DB.* TO fix_db@'%';

6. Load the schema into the new database created earlier. From within mysql run:

source mysql-fix-analyzer-v1.2-db-schema.sql

Or from the shell pipe the schema into the mysql command-line client as follows:

mysql connection_parameters FIX_DB < mysql-fix-analyzer-v1.2-db-schema.sql

7. Schedule the housekeeping event. In the following SQL, HH:MM is the daily execution time of the event and DAYS the number of days of message data to retain.

It is recommended that the housekeeping be run at a time of day where (ideally) no FIX sessions are operating or when no messages are sent. The installation script uses default values of 00:15 and 3 respectively.

CREATE EVENT FIX_DB_housekeeping
ON SCHEDULE
EVERY 1 DAY
STARTS DATE_ADD(CURRENT_DATE(), INTERVAL 'HH:MM' HOUR_MINUTE)
DO
CALL FIX_DB.removeExpiredFiles(DAYS);

Using the report

All FIX Analyser 2 dataviews except for the Admin view provide a right-click context-menu command ShowOrders… on rows. This command produces a HTML-based report giving additional details on the orders associated with the row, and optional message drill-down functionality assuming the MySQL message database has been configured.

To use the command simply right-click on a row and execute it.

Figure 23 - A "Show Orders" command being executed on a User View row.

Targets

/geneos/gateway/directory/probe/managedEntity/sampler[param('PluginName')='FIX-analyzeR2']/dataview[param('ViewType')='User']/rows/row/cell

/geneos/gateway/directory/probe/managedEntity/sampler[param('PluginName')='FIX-analyzeR2']/dataview[param('ViewType')='MessageDetector']/rows/row/cell

Arguments

Executing the command will display the following dialog. The number of results returned for a report is limited to maintain performance. By default the most recently updated 1000 orders are returned. The number can be changed as desired in the dialog.

Figure 24 - A "Show Orders" command order limit dialog.

When the command is executed in Active Console, the report will be displayed using the default browser on the client machine. Example report output is displayed below. Orders associated with the selected row will be listed in a table.

This table supports several options for controlling its content, such as the number of entries per "table page", or whether completed orders will be hidden (they are shown by default). The columns of this table can be sorted by clicking on the column headers, and the table content can be filtered dynamically by typing in the search field.

Figure 25 - An example "Show Orders" report.

If an order row is selected, then a FIX message drill-down table will be displayed below the order table. This feature requires the MySQL message database to be configured and available. The message table displays the individual FIX messages detected that made up the order, including chained messages where orders are changed or replaced.

As per the order table, the message table can also be sorted by clicking on the table headings. The raw FIX message contents (the full input log file line) is available by clicking the "Show/HideFIX messages" button.

Figure 26 - An example "Show Orders" report showing message drill-down output.

Customising report output

The columns in the order table of the report output can be configured using the reportColumns setting for both user views and message detector views. This allows fields to be added or removed on a per-dataview basis so that order reports for different views can be more relevant to the data being displayed for that view.

For example, three additional columns have been added to the default columns using the following setup:

Figure 27 - Gateway Setup Editor configuration showing custom order report columns for a user view template.

This configuration produces the following output:

Figure 28 - An example "Show Orders" report showing additional customized columns.

filters > filter > comparison

The comparison section of a filter marks the filter as being a comparison-type filter. This filter compares a FIX message field value against a set of pre-configured values.

Mandatory: Yes (if specifying a comparison filter).

filters > filter > comparison > fieldId

This setting specifies the FIX field id which will be compared for a comparison filter. The value of this field will be extracted to compare against the configured values. If the message does not contain a field with this id, then an empty value will be used for comparison.

Mandatory: Yes

filters > filter > comparison > comparison

Using this setting you can specify the type of comparison operation which will be performed for a comparison filter. The table below lists the available comparisons:

Mandatory: Yes

filters > filter > comparison > values

This setting specifies values which the FIX field value will be compared against for this comparison filter.

If configuring LESS_THAN or GREATER_THAN operators, only a single value needs to be configured. During comparison all values defined will be checked, however for LESS_THAN operators the value of the largest value will determine whether the message passes or fails the filter. Similarly for GREATER_THAN operators it is the smallest value which is important.

Mandatory: Yes - at least one value must be configured per comparison filter.

filters > filter > regex

Regex filters compare the raw message content (i.e. the un-parsed text containing the FIX message) against a configured regular expression to check for a match.

Regex filters require a minimum of a regex pattern to be specified. If the message text matches the regex, then the FIX message will pass the filter and be processed.

The optional invert setting reverses the result of the filter. If enabled (set to true) then a message which matches the regex pattern will not pass the filter.

Mandatory: Yes (if specifying a regex filter).

filters > filter > and

The andBoolean filter combines the results of its configured sub-filters. In order for a message to pass the filter, it must pass all configured sub-filters. If any one of the sub-filters fail to pass the message then this filter also fails. If no sub-filters are configured, the and filter will pass all messages.

Any number of sub-filters can be configured, and will consist either of a name-reference to another filter or be a new filter defined inline at the point-of-use.

Mandatory: Yes (one of and, or, not must be specified).

filters > filter > or

The orBoolean filter combines the results of its configured sub-filters. In order for a message to pass the filter, it must pass any one of the configured sub-filters. If all of the sub-filters fail to pass the message, then this filter also fails. If no sub-filters are configured, the or filter will pass all messages.

Any number of sub-filters can be configured, and will consist either of a name-reference to another filter or be a new filter defined inline at the point-of-use.

Mandatory: Yes (one of and, or, not must be specified).

filters > filter > not

The notBoolean filter inverts the result of the configured sub-filter. A FIX message which passes the sub-filter will fail the not filter, and vice versa.

The sub-filter can be either a name-reference to another filter, or be a new filter defined inline at the point-of-use.

Mandatory: Yes (one of and, or, not must be specified).

views > user

User views are a customised view configured by the user. These views contain configuration settings for the rows (categories) and columns (statistical values) to be computed for the input FIX messages and displayed in the dataview. Column values can be driven using order data or values from individual FIX messages as specified by the column type.

The user view can be configured either by specifying the view settings directly in the plug-in configuration, or by referencing a user view template static variable. If the view is specified as a static variable, the setup can be shared by a number of plug-in instances.

Mandatory: No

views > user > fix-analyzer-user-viewTemplate

This setting references a user view from a static variable.

views > user > data

This setting specifies a user view configuration directly in the plug-in setup.

Mandatory: No

views > user > data > viewName

Specifies the name of the user view. View names must be unique across all views for a sampler, so that each view can be uniquely referenced.

Mandatory: Yes

Row configuration

views > user > data > rowIdentifier

The row identifier setting specifies how FIX messages are categorised when computing statistical data in the output dataview. The values of the specified FIX fields are extracted from the FIX message being processed, and used to form the name of the row which will process updates from the message.

This process works slightly differently depending upon the column type as follows.

Message columns

For each FIX message processed, the value of the rowIdentifier FIX field(s) are extracted and used to generate a row name. If this row does not already exist a new row is added to the view. Message columns in the named row are then updated with the content of the FIX message.

If a message does not contain the specified FIX field (or has an empty value) then the value of the field as seen by the associated order for this message (if any) is used. If there is no order, or the order does not contain a value, then the text "<none>" is used as the row name.

Order columns

Operation is similar to the above, except that row names are generated directly from the order information associated with the FIX message. The appropriate row is found (and created if necessary) and order columns updated with any changes to the order state caused by the FIX message.

It is allowable however, that the first message of an order contains a value for the rowIdentifier field, and that the remaining messages do not contain this FIX field at all. In this case the field value will be cached and all FIX messages for the order correctly processed by the same dataview row.

Mandatory: Yes

views > user > data > rowIdentifier > custom

The custom row identifier allows the user to specify the FIX field tags, which will be extracted and used as the row name. FIX Analyser 2 allows 1 or 2 levels of identifier tags; primary and secondary. Secondary tags can only be specified once a primary tag has been configured.

If both primary and secondary row identifiers are configured, the result is a view with 2 levels of row name. Row names of the form primaryValue#secondaryValue are value rows, and the statistics (column values) on this row display values for those messages with matching identifier fields. Row names of the form primaryValue are summary rows, and summarise the data of the value sub-rows with the same starting prefix.

See the user guide configuration section for example screenshots.

Mandatory: Yes - one of custom or senderTarget must be configured.

views > user > data > rowIdentifier > senderTarget

The senderTarget setting specifies that the view should use the FIX fields SenderCompId (49) and TargetCompId (56) for the row identifier.

If using these fields you may also optionally enable the session status columns, which allows you to create a customised session analysis view.

Mandatory: Yes - one of custom or senderTarget must be configured.

views > user > data > rowIdentifier > senderTarget > showSessionStatus

If enabled, this setting adds the columns status, logon and logoff to the user view. These columns are normally displayed in a session analysis view, so enabling these allows the user to configure a customised session analysis view using the user view.

views > user > data > rowIdentifier > senderTarget > showSessionStatus > sessionTimeout

The session timeout specifies the default maximum period of inactivity for a session in seconds. If no messages are sent or received for this period of time, the session is considered to be timed out. This is displayed in the logoff column.

When timeout delay setting is non-zero, the maximum period is increased by the value specified. This setting can be used to cater for delays in writing or processing FIX messages which would otherwise incorrectly display a session as timed out.

Timeouts are calculated depending on the configured timestamp setting:

• If sendingTime is used, then the plugin converts the FIX Analyser 2 Netprobe server's timestamp to UTC, and then compares it to the value of the filed sendingTime of the FIX message with tag 52, which is also in UTC.

• if custom is used, then the plugin compares the captured timestamp to the value of the filed sendingTime of the FIX message with tag 52. The timestamp is not converted to UTC.

views > user > data > rowIdentifier > senderTarget > showSessionStatus > timeoutDelay

The timeout delay setting specifies an extra period (in seconds) of allowable inactivity for a FIX session. If a session is idle for a period equal to sessionTimeout +timeoutDelay seconds, then the session is considered to be timed out. This setting is ignored if timeouts are disabled.

Expected use of this setting is to cater for any delays between FIX messages being written to log files and being fully processed by FIX Analyser 2. Additional time can be granted to prevent the session being incorrectly shown as timed out; typically 1 or 2 seconds should be enough depending on normal message processing rates.

views > user > data > rowIdentifier > senderTarget > showExpectedStatus

This setting controls whether the expectedStatus column is displayed in the view. This column is normally present in a session analysis view if named FIX conversations have been configured. It can optionally be added to a user view to allow users to configure a customised session analysis view.

The expectedStatus column displays whether a session (senderTarget row) is expected to be Up or Down based on the time information configured in the conversation setting. Values in this column will be empty unless conversations have been configured.

Column configuration

The column configuration defines the data to be displayed for each row in the dataview output. Users can extract message or order values and compute selected statistics from these for display, using the configuration described below.

User views currently support two types of column; message columns and order columns.

Mandatory: Yes

views > user > data > columns > column > displayName

Specifies the name of the column as it will appear in the dataview. The name must be unique among all other columns configured for the view.

Mandatory: Yes

views > user > data > columns > column > rollingPeriod

This setting allows a rolling period to be configured for the column. This allows you to (for example) configure a column that shows a count of messages received in the last 5 minutes.

Values are specified as an integer in 3 different units: seconds, minutes and hours. To configure a non-integer period of 1.5 hours for example, configure the period as 90 minutes instead.

The period unit-type also specifies the granularity of the period. Values will be placed into "buckets" containing all values seen for a particular time. The time is taken from the FIX message timestamp truncated to the unit specified. The overall value of the period is then a statistical operation on the most recent X buckets (where X is the specified period length).

e.g. When using a period of minutes:

Therefore, for more accurate rolling period values users should choose a more granular unit type when only using a small number of units. For example, a rolling period of 60 minutes would be better than a period of 1 hour.

Mandatory: No

views > user > data > columns > column > filter

References a filter from the filters section by name. Filters are applied to input messages before the column processes any data. This allows the creation of (for example) a count of Cancel messages, by configuring a count column and filtering out all non-Cancel messages.

See the message column and order column sections in the Technical Reference guide for more details.

Message columns

views > user > data > columns > column > columnType > message

Message columns process FIX messages individually when computing their statistics. There are 3 types of message column which can be configured; sum, count and value columns.

If a message column is configured with a filter, then only messages which pass the filter are processed by the column. Therefore, a count column (for example) would count only the messages which passed the filter. This could be used to create a count of Cancel messages, by applying a filter that accepts only Cancel messages.

When aggregating an empty data range, the sum and value message columns will output an empty call in the dataview display (count columns will display as 0). This behaviour is intended and allows for improved data processing when used in conjunction with the database logging or charting features of Geneos.

Mandatory: Yes

views > user > data > columns > column > columnType > message > type > sum

The sum message column sums the values of the specified FIX field, for all messages processed by this column (i.e. messages that passed the filter, if configured).

Summary rows for sum columns will display a sum of all values displayed for the sub-rows.

A sum message column could be used to obtain the total size (in bytes) of FIX messages processed for a particular party, by summing FIX field 9 (BodyLength).

Sum columns will produce an empty value when no data is present. This may occur when summing messages that do not contain the configured FIX field, if a rolling period is configured and no values have been received within the configured period, or if the column has processed no FIX messages at all.

Mandatory: Yes - one of sum, count or value must be configured.

views > user > data > columns > column > columnType > message > type > count

The count message column is an integer value which starts at zero and increments by one for each message processed by the column (i.e. messages that pass the filter, if configured).

Summary rows for count columns will display a sum of all counts displayed for the sub-rows.

A count column could be used to could all Reject messages for a particular client, by configuring a filter to match on FIX field 35=9 (OrderCancelReject).

Mandatory: Yes - one of sum, count or value must be configured.

views > user > data > columns > column > columnType > message > type > value

The value message column stores the extracted FIX field or named value of the latest message processed by the column (i.e. messages that passed the filter, if configured).

Named property values which can be selected are as follows:

Summary rows for value columns will display the latest value (using the message timestamp) of the values displayed by the sub-rows. If no messages with the configured field are received (within the rolling period if configured) then an empty dataview cell is produced. See the message column description for a brief summary on this behaviour in the Technical Reference guide.

A message value column could be used to show the latest error for a particular session, by extracting FIX field 58 (Text). This field is typically used to describe the error for a Don't Know Trade (35=Q) or an order reject execution (35=8 and 150=8).

Mandatory: Yes - one of sum, count or value must be configured.

Order columns

views > user > data > columns > column > columnType > order

Order columns display a statistical summary for all orders applicable to the given dataview row. For example if the row configuration specifies that rows equate to exchanges, then order columns for a specific row will display statistics for orders to that exchange.

Available column types are:

• Value
• Sum
• Min
• Max
• Average
• Standard deviation

Order statistics can be filtered by the current order status, as specified by the match states setting. See the order tracking section in Message Processing for more details.

Filters for order columns are applied to the first message of the order seen for each dataview row, so that the order is only filtered once for that column.

Order columns (with the exception of the count column type) may return an empty cell value when aggregating an empty data range. This behaviour is intentional and allows for improved data processing when used in conjunction with Geneos database logging or charting features. See the specific reference section for each column-type for further details.

Mandatory: Yes

views > user > data > columns > column > columnType > order > type > matchStates

This setting specifies the states of orders which will contribute towards the column value. Orders with a state that has not been configured here will not be considered by the column.

Possible states are as follows:

See the order tracking section for more details on state changes.

For example, a count order column configured for the unacknowledged state would show a count of all unacknowledged orders. Orders which were subsequently acknowledged would then be removed from the count. By adding a filter to this count, we could (for example) count all unacknowledged orders of a particular type.

Mandatory: Yes

views > user > data > columns > column > columnType > order > type > count

The count column shows the number of orders which pass both the configured filter and order match states, or 0 when there are no matching orders. An example usage of this column could be to display the number of open orders (i.e. states "new" or "partiallyFilled") for that row.

Mandatory: Yes - a valid order-column type must be configured.

views > user > data > columns > column > columnType > order > type > value

The value column shows the latest updated value for the specified FIX field or named property, from orders which pass the configured filter and order match states.

Value columns will display an empty if no matching values are present. This may occur if messages for matching orders do not contain the configured FIX field, if a rolling period is configured and no order updates have occurred within this time, or if no orders were processed at all. See the order column description for a brief summary on this behaviour.

Mandatory: Yes (a valid order-column type must be configured).

views > user > data > columns > column > columnType > order > type > sum

The sum column sums values for the specified FIX field or named property extracted for all applicable orders, subject to the filters and order match states selected. Non-numeric values processed by this column will be converted to a number if they begin with a numeric prefix, otherwise they will be treated as zero.

An example usage for this column could be to sum FIX field 151 (LeavesQty) for all orders in states new or partiallyFilled. The result would be the total quantity for all orders, which are the units remaining to be executed (per dataview row).

Sum columns will display an empty value if there are no matching orders or values. This may occur if messages for matching orders do not contain th configured FIX field, or if a rolling period is configured and no order updates have occurred within this time, or if no orders were processed at all. See the order column description for a brief summary on this behaviour.

Mandatory: Yes (a valid order-column type must be configured).

views > user > data > columns > column > columnType > order > type > max

The max column displays the maximum value of all FIX field or named property values extracted for applicable orders, subject to the filters and order match states selected. Non-numeric values processed by this column will be converted to a number if they begin with a numeric prefix, otherwise they will be treated as zero.

A column of this type could be configured to display the maximum acknowledgement-latency for all orders of a dataview row.

Max columns will display an empty value if there are no matching orders or values. This may occur if messages for matching orders do not contain the configured FIX field, or if a rolling period is configured and no order updates have occurred within this time, or if no orders were processed at all. See the order column description for a brief summary on this behaviour.

Mandatory: Yes (a valid order-column type must be configured).

views > user > data > columns > column > columnType > order > type > min

The min column displays the minimum value of all FIX field or named property values extracted for applicable orders, subject to the filters and order match states selected. Non-numeric values processed by this column will be converted to a number if they begin with a numeric prefix, otherwise they will be treated as zero.

A column of this type could be configured to display the minimum acknowledgement-latency for all orders of a dataview row.

Min columns will display an empty value if there are no matching orders or values. This may occur if messages for matching orders do not contain the configured FIX field, or if a rolling period is configured and no order updates have occurred within this time, or if no orders were processed at all. See the order column description for a brief summary on this behaviour.

Mandatory: Yes (a valid order-column type must be configured).

views > user > data > columns > column > columnType > order > type > average

The average column displays the average of the FIX field or named property values extracted for orders, subject to the filters and order match states selected. Non-numeric values processed by this column will be converted to a number if they begin with a numeric prefix, otherwise they will be treated as zero.

A column of this type could be configured to display the average acknowledgement-latency for all orders of a dataview row.

Average columns will display an empty value if there are no matching orders or values. This may occur if messages for matching orders do not contain the configured FIX field, or if a rolling period is configured and no order updates have occurred within this time, or if no orders were processed at all. See the order column description for a brief summary on this behaviour.

Mandatory: Yes (a valid order-column type must be configured).

views > user > data > columns > column > columnType > order > type > stdDev

The standard deviation column computes the deviation of all FIX field or named property values extracted for applicable orders, subject to the filters and order match states selected. Non-numeric values processed by this column will be converted to a number if they begin with a numeric prefix, otherwise they will be treated as zero.

A column of this type could be configured to display the standard deviation acknowledgement-latency for all orders of a dataview row.

Standard deviation columns will display an empty value if there are no matching orders or values. This may occur if messages for matching orders do not contain the configured FIX field, or if a rolling period is configured and no order updates have occurred within this time, or if no orders were processed at all. See the order column description for a brief summary on this behaviour.

Mandatory: Yes (a valid order-column type must be configured).

views > user > data > columns > column > columnType > order > fields > fieldId

The fieldId setting specifies a FIX field id, which will be used to extract values from FIX messages for use by an order column.

Mandatory: Yes - one of fieldId or named must be specified.

views > user > data > columns > column > columnType > order > fields > named

The named setting specifies a property which will be extracted from an order for use by an order column. Available properties are as follows:

For more detail on the order status, see the order tracking in the Message Processing section.

Mandatory: Yes - one of fieldId or named must be specified.

Custom Order Report output

views > user > data > reportColumns

The report columns setting specifies the set of columns to display in the "Show Orders" command output when run for this dataview. If no report columns have been specified for a view, the default set of columns is used:

Mandatory: No

views > messageDetector

Message detector views extract and display values from "interesting" FIX messages. They do this by dynamically adding new rows to the view when a message is detected which matches a configured filter. Each row in the view represents a message which passed the filter.

The values displayed for the rows are determined by the column configuration as detailed below. The message detector view can be configured either by specifying the settings directly in the plug-in configuration, or by specifying a message detector template as a static variable. If specified as a static variable, the view setup can be shared by a number of plug-in instances.

Mandatory: No

views > messageDetector > fix-analyzer-message-detector-viewTemplate

This setting references a message detector view from a static variable.

views > messageDetector > data

This setting specifies a message detector configuration directly in the plug-in setup.

Mandatory: No

views > messageDetector > data > viewName

Specifies the name of the message detector view. View names must be unique across all views for a sampler, so that each view can be uniquely referenced.

Mandatory: Yes

views > messageDetector > data > filter

The filter setting specifies the filter used to determine "interesting" messages. FIX messages which pass the filter will be added to the message detector view as a new row.

The filter is referenced by name from the filters available to the plug-in. To filter on multiple "interesting" message types, combine multiple filters using a Boolean filter.

Mandatory: Yes

views > messageDetector > data > messageTimeout

The message timeout setting specifies the maximum age a message can be (in seconds) before it is removed from the message detector view.

If the difference between the current time (obtained from the local time of the Netprobe host) and the message timestamp is larger than this threshold, then the row representing this message is removed from the view.

views > messageDetector > data > messageCount

The message count setting controls the maximum number of messages which will be displayed in a message group. If the count it set to 1, then the group summary row in the dataview will be omitted.

A message group is a set of messages which share the same unique id. If the maximum limit is reached, a new incoming message will replace the oldest message in the group.

Row configuration

views > messageDetector > data > uniqueId

The uniqueId setting specifies how to construct a unique row identifier (row name) from a FIX message for display in the message detector view.

The uniqueId is made up of one or more id elements, each of which can extract a FIX field or named property from the message. If using more than one id, the field values will be compounded together separated by dashes (the - character) to form the uniqueId.

If several messages are found with the same id, the message detector view will display these all as sub-rows under a group row named with the id. This set of messages is then called a message group. The maximum number of messages per group can be configured using the messageCount setting.

Mandatory: Yes

views > messageDetector > data > uniqueId > id > fieldId

The fieldId setting specifies which FIX field to extract from a message. This value will then be used to form the uniqueId (the row name) in the message detector view.

Mandatory: Yes - one of fieldId, named or autoGenerated must be specified.

views > messageDetector > data > uniqueId > id > named

The named setting specifies a named FIX message property to extract. This value will then be used to form the uniqueId (the row name) in the message detector view.

Available properties are as follows:

Mandatory: Yes - one of fieldId, named or autoGenerated must be specified.

views > messageDetector > data > uniqueId > id > autoGenerated

The autoGenerated setting generates a series of incrementing numbers starting from 1. These values will then be used to form the uniqueId (the row name) in the message detector view.

Users should use this setting if they are unsure whether the FIX fields they have specified as identifiers will be unique. If using only autoGenerated to create a unique id, it is recommended to disable the message group feature via the messageCount setting.

Mandatory: Yes - one of fieldId, named or autoGenerated must be specified.

Column configuration

views > messageDetector > data > columns

This setting specifies the columns which will be displayed in this message analyzer view. For each message which passes the filter, field values from the FIX message are extracted as per the column configuration and displayed as a new dataview row.

Mandatory: Yes

views > messageDetector > data > columns > column > displayName

Specifies the name of this column as it will appear in the dataview. Column names must be unique amongst all other columns configured for this message detector view.

Mandatory: Yes

views > messageDetector > data > columns > column > value

This setting specifies the source of the column value. Choose between Field id and named.

If a FIX field id is specified, then the value will be extracted from the FIX message. If the specified field does not exist in the message an empty value will be displayed.

Users may alternatively specify a named FIX message property rather than a field value. The available properties are as follows:

Mandatory: Yes

Custom Order Report output

views > messageDetector > data > reportColumns

The output of the "Show Orders Report" command can be customised for message detector views, in the same way as for user view. Please see the user view custom order report section in the Technical Reference guide.

views > sessionAnalysis

The session analysis view displays the current status for the conversations on a monitored FIX engine. This view is useful for ensuring that specific connections remain up throughout the trading day. For more information, please refer to the section on session state tracking.

Each Fix-analyzer 2 plug-in can output a single session analysis view, which is created when this setting is enabled.

Expected FIX conversations can be configured using the conversations setting. This also allows other properties such as an alias to be specified. A start and end time can be specified for a conversation during which time the session is expected to be up.

The session analysis view can be configured either by specifying configuration directly in the plug-in configuration or by specifying a session analysis view template static variable. If specified as a static variable, the view setup can be shared by a number of plug-in instances.

views > sessionAnalysis > fix-analyzer-session-analysis-viewTemplate

This setting references a session analysis view from a static variable.

views > sessionAnalysis > data

This setting specifies a session analysis view configuration directly in the plug-in setup.

Mandatory: No

views > sessionAnalysis > data > viewName

Specifies the name of the session analysis view. View names must be unique across all views for a sampler, so that each view can be uniquely referenced.

Mandatory: Yes

views > sessionAnalysis > data > showSessionStatus

The sessionTimeout setting specifies a default timeout period for a FIX session conversation. This period will be overridden by the timeout (if specified) of the corresponding named conversation if available.

Combined with the timeoutDelay setting, this defines a default maximum period of (sessionTimeout+timeoutDelay) seconds of inactivity for a session. If no messages are sent or received for this period of time, the session is considered to be timed out. This condition is displayed in the logoff column.

Timeouts are calculated depending on the configured timestamp setting:

• If sendingTime is used, then the plugin converts the FIX Analyser 2 Netprobe server's timestamp to UTC, and then compares it to the value of the filed sendingTime of the FIX message with tag 52, which is also in UTC.

• if custom is used, then the plugin compares the captured timestamp to the value of the filed sendingTime of the FIX message with tag 52. The timestamp is not converted to UTC.

The expected use of the timeoutDelay setting is to cater for any delays between FIX messages being written to log files and being fully processed by FIX Analyser 2. Additional time can be granted to prevent the session being incorrectly shown as timed out; typically 1 or 2 seconds should be enough for normal message processing rates.

Attended Scripted Upgrade

A schema installation / upgrade script is provided in the resources/db-schema folder of the FIX Analyser 2 package. The script should be run from within this folder and will require database administrator (DBA) access to the database. If remote DBA access has been disabled, then the script will need to be executed on the database server host itself (or remote access granted).

To execute the script to upgrade an existing FIX Analyser database, run the following command:

cd resources/db-schema
./fix_analyzer_schema_install.sh -u

The script will prompt for the following information during the installation process:

• Database administrator (DBA) username
  • This account will perform administrative functions on the database
• DBA password
• Hostname for the MySQL database server
• Port for the MySQL database server
• Name of the FIX Analyser 2 database to update
  • This database must already exist on the server
• Time to run the daily data housekeeping
  • Default time is 00:15
• Number of days to retain message data
  • Default is 3 days of data

Unattended Scripted Upgrade

The upgrade script described above can also be run in unattended (non-interactive) mode.

To do this, the requisite information the script requires must be provided as environment variables. The table below describes the environment variables the script expects when upgrading an existing database installation.

Manual Upgrade

If you are unable or do not wish to install the database schema using the installation script, you can install the schema manually using the following steps. The SQL statements below should be executed as a database administrator (DBA) account on the MySQL database server.

1. Connect to the MySQL database FIX_DB using the MySQL command-line client. See the reference (http://dev.mysql.com/doc/refman/5.1/en/mysql.html) for full details.

mysql -u dba_username -p dba_password [ -h db_host -P db_port ] FIX_DB

2. Enable the MySQL event scheduler. Note, you should also enable the scheduler permanently as detailed in the server installation section of this document.

SET GLOBAL event_scheduler = ON;
SHOW PROCESSLIST;

The output of the SHOW PROCESSLIST command should return a row with the text event_scheduler if it was successful.

3. Verify the current schema version.

SELECT * FROM version;

4. Upgrade the database schema using the upgrade SQL files. To do this from within mysql use the command:

source sql_file

Or from the shell pipe the schema into the mysql command-line client as follows:

mysql connection_parameters FIX_DB < sql_file

The exact files to use will depend on the version of database being upgraded. For a version 1.0 schema, first upgrade to version 1.1 using the SQL file:

mysql-fix-analyzer-v1.0-v1.1-schema-upgrade.sql

Then, upgrade from version 1.1. to version 1.2 using the SQL file:

mysql-fix-analyzer-v1.1-v1.2-schema-upgrade.sql

5. Schedule the housekeeping event. In the following SQL, HH:MM is the daily execution time of the event and DAYS the number of days of message data to retain.

   It is recommended that the housekeeping be run at a time of day where (ideally) no FIX sessions are operating or when no messages are sent. The installation script uses default values of 00:15 and 3 respectively.

CREATE EVENT FIX_DB_housekeeping
ON SCHEDULE
EVERY 1 DAY
STARTS DATE_ADD(CURRENT_DATE(), INTERVAL 'HH:MM' HOUR_MINUTE)
DO
CALL FIX_DB.removeExpiredFiles(DAYS);